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CHAPTER 1 


1.1 (a) W(p) = - p&np - (1-p) &n(1-p) nats. 
2 

d i=p 1 d - 

a m= & = Q ==> = — d — = QO. 
dp % (P) ee Pp => an i 7 HC (p) HCD) 
; 7 

(b) w P H(@,) = og | ioe 
2 9) = P log 2 BUF) 182 a5) 
2 
at P + D(l=p) log Sea) = G5) cloe 5 
(1-2) 
ul al 
aia }- = 2] iL — + ino —_—_— 
1°2 P(I-p) p log 5 + (1-p)log 7 
i ae, = 2 #(p) 
2 
an45 (1=p) 
te) 2) HG) log 52 8.5.7 bits 
ii) H@ = a 13 =.3.7 bits 
13. 10 pis F 
iii) H@ = 73 1°8 3 + = 13 Bo) cee iex vi? bite 


(d) i) H(fair die) = log 6 = 2.58 bits 


it) PCk) = KC ke 12 3.4.5.6 
6 6 
>> P(k) = ee a 
k=1 k=1 
6 
— 41a =D FE log = 2.39 bits 
k=1 


1.2 Inequality (1.1.8) gives 
Han... a?) “2 Py(u) log ST AO 


< 
S Py (u) log Lo - (= iff QC) = PLC) 


Choose Qy Cv) = TT a (u,) where 
n=l 


eee 2s ye 212 
itn u, 


are the marginal probability distributions. Then 


py ab) N) x 1 
B(2bP... ah <Eyo( Y log ser 


-E(Es, (u) log SC ) 


(u_ ) 
>> ie (u) log 5 
n=l1‘\u Pr (u) 
= Hae”) 
n=1 


ith Gy tee ore ye 47 (rH) 
with equality i yw = ae (un) 
n= 


1.3. (a) Given discrete random variables x,y € Z®xwYW with joint 


probability P(x,y), we can define marginal probabilities 


P(x) =) P(x,y), P(y) =)>P(x,y) and conditional probability 
y x 


P(y/x) = P(x,y)/P(x). Then using inequality (1.1.8), 


1 
H(Y |B) =D) P(x, y) log 
— P(y |x) 


i 
=) P(x) P(y|x)log ) 
Erie Ze0ee rds 
< D700 (TP lo108 55) 
x y 
(XL PwPcls) 108 ae 
x 


x 


H(y). (1) 


(Ld, 


Now fix N and consider sequences uEeW =U UM) where 


k 
us ) = %@is the alphabet of the kth term in the sequence. Using 


2 


the relations 


PyCuy uy? = Play Uy. UNeD? Py-1°"1,...,"n-D 
N 
= TT Pt, ae 
n=2 
we have 
H(2¢,) = HQy_,) + nla BY. WO-D) (2) 
and ; N — 
nary = Hl) + yond [kD x, WO) (3) 


n=2 


Note that from (1) we have 


A? nD) «wh fale, ) 
= (BY LSB see 2en-2)) (4) 


where the second equality comes from the stationary property. Hence 


(a » 


(3) is bounded by 
HM) > xi (ab [ah i xUOD) (5) 


Using (5) in (2) gives 


HO) < HOH 1) + +H). 


or 


ji H(%4,) < = HMA _1) - oe 


Then 


H@%) . H@%,) 
ook 


for - <= a. 
n Ph 


(b) This follows directly from the proof of Theorem 1.1.1 


when we define 


S(N,€) 


{uy . Nite] -N[H-e] \. 


< Py = 


and use lim F 


N O tt0n (3.175 5)) te 1114s): 
N>0 


> 
oe 
~ 
Ne 
Q 

tt 


; E{ (x-m) 7} = E{ (x-m)” | |x-m| > e} Pr{|x-m| > e}. 


+ ECan)” | |x-m| < e} Pr{|x-m| < e} 
E{ (x-m)* 


|v 


|x-m| > e} Pr{|x-m| > ec} > e*Pr{|x-m| oe), 


N N = 
(b) Pr{Z ys. y ees Z+e} < Pril= Dz = Z| > es oat 
oN ee oe BEE a Z 
n=1 n=1 € 
2 
ae 
ne’ 


N 
as [20 207 Nite) | 
1.5 For ue S(N,€), we have 1 < 2 re ~ 6 > O. 


Hence, [X vA Nite) | 
Dea PP ayaa 
ve S(N,€), ue S(N,€) E) 
N 
s| > Z-N ce) 
3 OP — 
u 
$s | > Z_-N(Hte | . 
_ a n=1 \ ies [Pre] 
n=l 
NG) 
where G(s) = s(Hte) - log nfo? 
- 1-s 
= s(Hte) - log} > P(a,)**} 
=] 
and G(0) = 
Next note that a ‘ ‘i 
-$ 
eG 
deG) = el 
ds A 1% 
2d, Pla) 


where 


ds 
$=0 
and 
2 A ok ; 
eae) 5a, ( fi rep) - ( DQ (a,) log 4. | 
a ae a P(a,) ~*s k P(a,) 
<0 


since this is negative of the variance of log Foy with respect to 
distribution 


oy Gey Gas 
A 


BC ae 
k=1 


Qs (u) = 


Thus, there is a unique maximum of G(s) for some s*> 0 where 


* 
G(s*) > 0. Then we have F< 2 NG(s ie Similarly we get 
= a7 ae = 
Fy < 2 NG*(s ) where Ca) > 0. then 
x: ~ KK 
on a -NG(s°) -NG(s*”) 
fet ee 2 +2 


-6 Multiply the inequalities 


1 1 
log PG) < &(u) < log Pa) 71 


by Pf) and sum over all uEM, Then 


H(Wy) < <Ly> < H(@%,y) + 1. 


N 


But from (1.1.14), H(%@,) = NH(%) giving us (a) and (b). To show 


-2(u) 


(c) note that in (a) the inequality log Hey = L(u) ——>_2 Fa) 


Hence we satisfy the Kraft-McMillan inequality, 


Staton e255 Pw), 2 1 
u u 


The solution is best understood in terms of a tree diagram where 
left directed branches correspond to "0" and right to "1". Each 
node corresponds to a binary sequence so that code words can be 


represented as nodes in a tree such as 


QO <— 1 


000 001 100 101 110 a oe & 


If node (a) is selected as a codeword of length 2, then in order 
that no other codeword have 01 as a prefix no nodes that branch out 
from node (a) can ever be se]ected as a codeword. Hence we can ter- 
minate the branching at this node. Thus uniquely decodable codes 
with the property that no codeword is a prefix of another codeword 


corresponds to nodes in a tree where no codeword node branches out 


from a shorter codeword node. In general, let his Lo sereshy be the 
A 

set of codeword lengths where hy K hy <1 K hy. If >> 7 ee ee 
i=1 


then we can easily find such a code with these lengths where all 

branch paths of the code tree terminate in a codeword node. If 
A 

>y 7 ac < 1 then some branch paths can continue forever without 
i=1 


encountering a codeword node. Let xX) be any node sequence with hy 


branches leading to it and denote it the first codeword. There re- 
main ree os unterminated nodes at the same level that can be a code- 
word or a prefix of a codeword. If 2) = Ly choose any one of these 
remaining nodes as the codeword node of sequence x Lt 2) > wal 

then use any one of these as a prefix and proceed along any 2) - hy 
additional branches to find a codeword node of There now remains 
(2°4-1) ota 4) nodes at the level ho that can be a codeword or a 


prefix of a codeword. Continue in this manner until X, is selected. 


If at any point this procedure cannot be completed because of no 
A 


remaining nodes then ye “Fe > 1, which is a contradiction. 
i=1 
P(y,x) 
ue U2:y) = P(y, aK 
abla (a) 1(D3y) 22, (y,x) log P(y) q(x) 
= kG pice yD Py.) Loe 2 
q (x) : q(x] y) 
Dee 3 + fae 
ee “~ ion Pca ae 
T I 
H(2) H(2|Y) 


But H(Q|Y) > 0 (see 1.1.9) so 1(M;y) < H(2) and by symmetry 
I(@Dsy) < H(Y). 
(b) Use P(x,y) = P(y)q(xly) in 


A 
H NX, = ' ees 
(B,Y) 24 LPC y) log PGs) 


ui 1 
L 2 P(y) + 2, UPCuy) log acx]y) 


=H(Y) +H(2|Y). 


1.8 (a) q Oo i-P o 0 Pp(0) = q(1-p) 
E PCE) = P 
l-q 1 AP, 1 P(1) = (1+q) (1-p) 


So 1-p P aii B 4 
I1(@®3;%) = ql pros (ae s)+ qp log (5) (1-q)p 108 (E ) 
l-p 
+ (1-q) (1-p) log ( (1-q) =) 
= (1-p)W(q) 
q = + maximizes #(q) so C = 1-p. 
(b) a 0 2 4 0 po) = 4g 
1/2 
eis) a o 1 P(l)=1-F4q 
i i 
(23sy) = + q log > +54 log a + (1-q) log 5 
ls a8 me 
E i oe ae 
=) 4: 208 3 a (lq) tog ‘= 


Hence C = log 


1.9 Since the encoder keeps sending the information symbol until an 
unerased channel output is achieved there is no error and - = 0. 
The probability that n channel symbols are transmitted for an infor- 


mation symbol is Pa = G-p)p. The average length of a codeword 


is thus = 
— 1 
L = > nP = a n(1-p)p™ > 1p 
n=] ne) 
Thus R =-i=1-p bits per channel use is the rate which also equals 


the channel capacity (see problem 1.8a). 


oe. -i 
1.10 There are two coins C, and C, where P(H|C,) an and P(H|C,) i 
Here q(C,) = q(C.) - 5 are the probabilities of selecting each coin. 


1 
We can interpret this as a BSC with p 77.88 follows: 


i 3/4 ee 
9 Cy re) a Go 8 FU) = 9 
1/4 

; i 3/4 ou 

yg 3/4 . Sus 

(a) I(C, 3H) = log 1/2 = log 5 bit 

ay 1/4 . pte 

1(C, 3H) = log ‘Wree log 5) 1 bit 


(b) 1123 yw) = 1 -3¢(F) bits. 


1.11 (a) There are 13 ways one coin may be heavier, 13 ways one coin 
—_—e 

may be lighter, and the possibility that all coins weigh the same. 
Thus we are attempting to determine one of 27 possible situations 
using a balance and a known standard coin. We are thus asked to ob- 
tain at most log 27 = 3 log 3 bits of information on the average. 
Each weighing has 3 possible outcomes (left, right, balance) and 
provides at most log 3 bits of average information. Two weighings 
has 9 possible outcomes and at most 2 log 3 bits of average informa- 
tion. Three weighings has 27 possible outcomes. Clearly two weigh- 
ings cannot guarantee determining one of 27 possible situations 
while with three it may be possible. 

(b) The maximum amount of average information from three weigh- 
ings is log 27 which is achieved if all 27 weighing sequences are 
equally probable. This means that we must choose a weighing strategy 
where the outcomes of each weighing are equal probable and each 


weighing outcome is independent of other weighing outcomes. Clearly 


each weighing must reduce the number of possible cases by 1/3. 

Strategy: Let S denote the standard coin, C denote a coin that can 
be heavy, light, or normal, h denote a coin that is heavy or normal, 
and 2 denote a coin that is light or normal. We start with 13C and 


ano. 


lst Weighing: Set aside 4C and place 5C on the left pan and 4C + §S 


t Balance => 4C remain 


Left = > 5h, 42 remain 
A Right => 5%, 4h remain 
is] & 4c +5 


Note that for each of the three possible outcomes we have 9 remain- 


on the right pan. 


ing unknown possibilities to be resolved with two more weighings. 


2nd Weighing When Ist Outcome is Balanced: Of the 4C set aside C 


and place 2C on the left and C + S on the right. 


Balance ==> C remain 


— wa Left ==> 2h, & Temain 


Right ~=> 22, h remain 


2C crs 


Note that here each outcome leaves only 3 remaining unknown possi- 
bilities to be resolved with one more weighing. The third weighing 


given the 2nd weighing outcome is as follows: 


10 


(i) Balance: Place C on the left and S on the right. 
(ii) Left: Place h on the left and h on the right. 
(iii) Right: Place 2 on the left and & on the right. 


2nd Weighing When lst Outcome is Left: Of the 5h and 42 remaining 


set aside h + 22 on the left and 2h + & on the right. 


, 


Balance => h, 22 remain 


Left => 2h, & remain 


Right == 2h, & remain 


2a ee an = % 


One more weighing easily resolves the 3 remaining unknown possibili- 
ties by placing the same type of coin on the balance. That is, if 
2h, & remain then place h on the left and h on the right. 

(c) Without a standard coin we cannot always reduce the number of 
unknown possibilities by 1/3 with each weighing. This is a neces- 
sary requirement. 


1.12 Using (1.1.8) we have 


ay es 1 et 
H, ¢ 2) poe P, @) ae Gye ee 


a 


Hence AH, (4%) + (1-)) H, (@%) 


JA 


1 
ae [Py ce + (1-)) P, (u)| log P, (a) 


u 


1 
= Puree P, (a) = Hy (4%). 


2 
ee Ae 
2 
- l 20 
1.13 Let p(y) = e Y . Then (1.1.8) gives 

V> 2 
TO 
y 


11 


pee 4 
Sore 55 PG) dy fron 2. ae: a Sr Pes log e + zoe en, ay 
7 


—- i 2\_1 2 
= 7108 e+ 7 LOS (270% | 7108 (2nec? 
with equality when p(y) = p(y) or y is a Gaussian random variable. 


For the additive Gaussian noise channel we have 


_ (y-x)? 


20° 


Ab 
(y|x) = —=—e 
P V 2n07 


With input probability density q(x), the average mutual information 


is “ 
(2; Y) = frmroe 5 Fyco) dy S00} fpolrs 56 Pox) | dx 
But 


Cc © 2 
- 1 2 
S polnies PICA ES) dy = froin| + Slog (2m70 |e 


= Flog (27e0°) 


Using this plus the above bound we have 


(M3: yw) < Mice (2neo, | - Flog (21e0") 


2 
P 
1 fo} 
= slog gue SE 
2 o* 


with equality if and only if y is a Gaussian random variable. It is 


Gaussian if 


2 
A Sr 
1 2é 
q(x) = Ara 
Vv 21€é 
with the result that es E+ o”: Hence 


12 


2 
max 1(23;%Y) = Flog (“3 ) Flog (1 + £) 
oO 


which is channel capacity. 


1.14 From problem 1.6(b) we have inequalities 


- log Pfu) = i(u) < — 10g PCy) + 2. 


Thus 
- log P,(y) a 
L(u) = 
and 
~ log PG) f(a) - 1 
- > =l- Ls a Helen = 
oa ug tee TG) 
But 
P_(u) = - Pia) < . P(u") = P( oo 
nie nae u |G (u") = P(u 
n=1 n= 
and 
log Pfu) < N 10g P(u*) 
Hence 
; - log P,(u) 
aa : as ‘au 
N log P(u*) x 
and 
1 + 1 < Hy. <1. 
N log P(u*) 
- log P(e) 
If H. = 1 for all N then = ] for all N. 
N Xu) 


Hence Py (w= 27) for all N which is possible only if Pfu) = oN 


and the source is a BSS. 


13 


2.15. For a €-S(Nie), 


NiH(@/) = ci < £(u) < N(H(@7) + ej] +1 


and 
N[H(@) - €] < - log Py Cw) < N[H(@) + €] 


Thus: tor-u ¢€ S(N,€), 


- log P._(u) 
N[H(@) - €] N‘~ 
1 + N[W(w)+ e] ~~ _2@m) <7 : 


Ror 1-6 Sth 6). 
N log A < &(u) < N log A+1 
and 


- N log P(u*) < - log Py fw) < - N log P(n*) 


where Pai’) = min P(u) >.0. Thue for uw ec SNe), 
ree: u 


= log Pius) 2 een © ~_N log P(u™*) (2) 
N doc A + &(u) a N log A 
Define Fy = Pr \u € S(N,e) \ and using the form 
- log P._(u) 
a. = P..(u) Ne 
Hw (LEE 
- log P, (u) ~ log Py (u) 
=> Py) ( Ru) )+ 2. ro RG) 
4 é S(N,é) WE S(N,8) 


we have from inequalities (1) and (2), 


N[H-e] 
Hy st 1 + N[Hte] 


- N log P(u*) F 


C1-F,) + NlogAt+1 N 


and 


- N log P(u**) - 


< pat 
Hy <Q Fy) 2 N log A N 
kk 
-1- 47, - et 7 1. 
og N7- © 
Hence for any e€ > 0, 
H-€ 
< lin <7 
BP e a So i 
or 
lim = 1 
eae 
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CHAPTER 2 


The noise compo- 


nents in the per- 


pendicular coordi- 


nate directions are 


independent with 


variance N,/2. 


Let q = Pr{n>a} 


VE 4) 


<a,-a<n 


2.1 (a) | : | 
{ 33a - ; N 
x | Xet X,1 Xx, 
eee i aS Se ae + a 
| 
a- 
Ac | X er X 
ah ened | | i ti on ensies 
T a ; T | T 
-3a | -a a | 3a 
ee 
9 | 10 se | Xi 
Re ie ees ze ee + MELO ke 
| 
{23 alte 
oe | ay ae ae 
l | 
(a) 
P= re a Pe Pe = Pr{n,<a,n,<a} = a eey 
Ce ae ie 
P, =P, =P, =P, =P =P, =P. =P = Prin 
C, C3 C. Ce Cy Cho a Cis ui 
2 
P. =P. =p = Pp = Pr{-a<n_<a,-a<n,<a} = (1-2q) 
Ce Cc, C10 Chi a Z 
Hence 


and 


(b) 


| -q)* + 2(1-q) (1-2q) + (1-24)? 


I 
le 


9 
P = 34-70 


Rotation and translation does not change P 


16 


2 

Po = ag [4(1-q)? + 8(1-q)(1-2q) + 4(1-2q)7| 
i 
4 


| (aa) Di (1-2q) |? = (2-33)' = ( an 


ag 


El 
2 


) 


=> 
we é 
Te 
tires 
9<a} = (1-q) (1-2q) 


2.2 (a) 


(b) An oe tys | |y-xeal |” < lly-x, 1 For all m'# mf La eT, are 


the optimum decision boundaries. We have 


an e pak 2 = 
ag {y: lly-x, | = Vé/2}.¢ A, m | By a ge 


Hence 
PE = PrtyéA |x} < Prf{yés |x} 
oes 
: Pr{nvin, > é\. 26 ae 12 
1 > 4 
é 
and -——_— 
a1 aN 
PS = 5 PE <e 0 
m=l1 m 


be 


2. PE < 2 P,, (mpm! ) where 
ae mnt 4m 
en ba Ee : : 
P_ (mem = ere we have 
E 
V 2Np) 
[Is,-S ll =v@ =m" = 2,3,..:,7 
eect le Vé ot = 3,7 
[|So-Sel| = V3e mm’ = 4,6 
Iso-S || = 2vé at = 5 
é 
Pe <P (om') = 6 Q E) 
1 mn! +1 
ee é 3é Ze 
*z < 25 ©. Gaul) 2a(\x- J+ 20(\E}ral 22) 
m! +2 
By symmetry Po. =P = Po = Pp fm Pep ie 
E, Ey E, E. E« E, E, 
Hence 
é 
Posto < 6 a(\/e-). 
E E, _ 2Np 
é 


GN 
From problem 2.2 we have P <e 0 which is "exponentially" the 


same since Q(x) is bounded as shown in (2, 3.28). 
— ae = 
2.4 (a) Choose o 6b) = a x(t) m= 1,2,..., M as the orthonormal 
basis. Then x_ = VE6 mon = 1,2,..., M and ||x Ve = @ for all n. 
mn mn. -m 


The decision boundries become 


Z 2 
Ne = Ay: | iz | <| ly-x ||“ for all m' 4m} 


ty: (yox)>(y ox) for all n! +m} 


: ‘ ae 
{y: y-77e tor all mo +m} m= 1,2,..0,M. 
Hence by symmetry, 
Po Sr = i= Po a7 Prty,>y., for all mt1|x,} 


a y 
18 


(b) Given x 


independent random variables sb Vy given by 


_ g-vey* 
> ty) = i e “o 
Yq VT™No 9 
“Fy 
if 0 
p. (ty) = e a= 2.3. 4. Mu 
Yn Vv To 


co 


= < = 

fo, a for all m1|x,,y,-0} p, (a)do 
—OO 1 

(c) Using & = é logo, the fact that as € gets small I1n(l+e) 


2 
behaves as €, and Q(x) behaves like a /2 for large x, 


lim In| 1- a( x+/2 &, logM/N ee 
M0 0 


lim (M-1) ln [2- Q(x v2 E,logM/N 5 )] 


M>co 
= -lim M Q(x ve logM/N 
ae b 0 
a 2 
= -lim M exp j- 5 | xt ve) E losM/N, 
Meco 
& log 
- lin 1 655: 
Meo ay 
eee: 
Npin2 
= -lim M 
M->0o 
& 
= 0 
é 
ee < > 1n2 
0 


was sent we have probability density functions for the 


Hence 


é 
0 ; mil ae bo 
M-1 No 
lim EE Q (x+v2 EIN )) - é 
ie 1 : ma a ine 
0 
But 
Priy <q for allm#i|x, .y,=01 
M 
= Ty Prey soley 
me2 
M-1 
cl 
N/2 ; 
Hence oe on VE 
| ees “fre le = e “o da. 
E ip No/2 Vv 1Ny 
=—CO 2 
es x 
-— M-1 
Se ee es AP ee a(x ze | dx 
J 27 -—o 0 


2.5 (a) K=1=—> x, Werett 11, 5 = Vé/2 [1 -1] 


and (x,4%5) = (€/2) [1 1] LT} 0. 


(i) th Ob th 
Let bed denote the i row of Ay and hy the i row of Hy - 


The codewords of length M = 2X are given by 


. K-1) 
nD - E G) no) and (#2) pa? 46] a 
Suppose a), he) m0 for: atl 1 + 4 < 9k-1 
Then 
(i262. no) =o sor an sts ck 
and 
(xf 42. 92 AO) = ovr args 


20 


yields 
(a 1) «0 for all 1% 4 < ga 
(b) Note that (KX. nf?)- C544: Hence if we subtract the hah 


product term of this inner product we have 


(c) Let a = /&/M [1 00... 0]. Then the orthogonal signal set 


1? 


translation of the signal set given by, 


\ and the simplex signal set are essentially related by a simple 


ces cm eee ae 


oe ok 


(Since the 1°° component of x 


Ux > 


~ 


is always "0" we can ignore it.) This 


i 
means the error probability of the orthogonal signal set of energy 
M 
soaties] is the same as the error probability of the simplex signal 
set of energy &'. 
M 
(4) Let W= 2) x,- Then, 
i=1 
0< WW) = D> (x5 >%,) 
a.) 
= > (x, »*,) "GD (x5 08,5) 
i j=i itj 
=M&+), >, (x, x4) 
itj J 


or 


2, (x55) a ee 
itj 
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or 


1 
ye eee ep Bre a 2 
Ses itj S tes 


aA 


(e) Let ZyoZoorres By be an orthogonal signal set of energy &. 


Then ES, = &6,,- 


Let 


M 
ae 


Ei 


and consider the translation set formed by 


aa _ 43. = 
Then 
(y »Y,) = So ra 
= (2492, ) Ne (Z5 »a) = z, a) ~ (a, a) 
= &§ ; -~a&/M - a&/M + a* &/M 
; &(1- ~20,/Mero" /M) 
[2 E (9.720) /M 
Choosing 
€ and a to satisfy the equations, 
A 2 
& = & (1-2a0/Mto" /M) 
and 


Pi 
Ep =B(a°-20) /M , 
yields a signal set ty, } with the same energy and inner products as 
the signal set {x}. It then has the same error probability and 
é 1- 
alte 0) = 2 (29-0) a). 


No No 


mW uae 
De Ge) nt) of dee ee | eae 
= a W ~ 2mW jt TWt 


—TW 
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z(t) y(t) y(q) 


— h(t) ag Wah 
tai 
Z P sin[1W(t-1)] 
a a | 7 /gin[mWtt-7) 
y(t) = fnce-n2(n) ar = f SERED] (yar 
0 0 
n 
and ~ sin mw(2 ~t) mi sin nw(t- 2) 
y(2) -f L . ] z(t) dt - few 7 le 
mw(B -) TW (t- 9] 
0 W 0 W 
(c) ‘In Figure 2.9, y(t)®,_ (t) is integrated over [(n-1)T/N,nT/N] 
beta 2 sin|1W (.- =] 
6, (t) = sin WT. 
2n ( z 0 
TW {t- — 
W 
Thus nT/N ~ ( S a) . 
ae pe sin| 7 aa ] 
Qn Vit) v2 sinw,t = dts. 
(n-1)T/N uh (¢ - =) 
Replace the integration over [(n-1)T/N,nT/N to [0,~) we obtain 
ne sin mw (t - al 
ee “  O 2 sin Wot dt 


“e-3) 


0 W 


which is the process in Figure 2.11. 


2.7 (a) Here y(t) = ¥ 9, 6b) where o(t) = /2N/T sin Wot for (n-1)T/N 


< t < nT/N whereas we compute 


nT/N 
oy -[ y(t) $)(t)dt 
(n-1)T/N 


where 


o* (t) = J2N/T sin[w,+hw) (t-(n-1)T/N) + 9], 


(n-1)T/N<t<nT/N. 


Se 


T/N Wot 
Using SJ 028 [Quytiu)tto]at = 0 since — >> 1 


N 
0 nT/N 
we have ss vn JS op (t)dt 
(n-1)T/N 
nT/N 
om ys (2N/T)sin wot sin[ (w, thw) (t-(n-1) T/N)+9] 
(n-1)T/N 


T/N 


fe S nity sin Wot sin[ (wtdw) tt ]dt 
0 


T/N 
y,, (N/T) S {cos (hut+9) ~ cos[(2w)tdw)t+o] }dt 
0 


T/N 
a win f cos (Awt+) dt 


0 
T/N 

yout) f {cos Awt cosd - sin Awt sing}dt . 
0 


We assumed here that WoT /N is a multiple of 7. Also since AwT/N << | 


I 


we have sin Awt = 0 for O<t<tT/N. 


Hence 


T/N 
vy. = y,(N/t)cos¢ J cos Awt dt 
0 
= sin (AwT/N) 
aa ( (AwT/N) 
(b) Here nT/N 
ye - S ye) V2N/T Sin (Wott) dt 


(n-1)T/N 
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nT/N 


ery = S yt) y¥ 2N/T cos(wpt+>)dt 
(n-1)T/N 
where 
1) = ey iol bon t) ‘ Seat! Ponty 6t) 
nT/N 
Thus ve = Xo, Sanity sin Wot sin(w tto)dt 
(n-1)T/N 
nT/N 
+ X41 J (N/T) cos Wot sin(w t+) dt 
(n-1)T/N 
nT/N nT/N 
= x5, (N/T) S 080 dt + X41 (N/T) yA dt 
(n-1)T/N (n-1)T/N 
= Xp ,CO8d + Xon4] sing 
Similarly 
Yontl isn cee mos baat : 


(c) The four possible phases are given by 


2) hifi). 


Suppose Xon =Vé, Xont] = 0. Then the signal components in the ob- 


servables become Yon =V&, cosd, ey =/@, sin >. 
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The error probability is the probability that the noise components 


cause the observable vector to lie outside the correct decision re- 


gion. Hence 


P,=1-P,=1- Pr {n, <dj,n 


E C < dy} 


Zz 


1 - Py {n, < d,} Pr {n, < d,} 


[ial al)p-al 4) 


I 
— 
Fe 

fs 
ao 
© 
— 
a 
io) 
Hele 
—_ 

I 

2) 
EET 

2p 
(2) 

aa 
—— 


ll 
; B 
—_ 
ait 
i) 
Q 
(o) 

n 
—«— 
A 
I 
oe 
eee” 
—— 
+ 
PET 
a 

© IO 
n 
nee 
fe | 
caster 
oS e| 
i] 
Co 
—— 
—— 


2.8 
%s (a) The boundries 
a 
/ rd x a8 of the optimum de- 
cision regions are 
N shown as radial 
: VES a lines. Each deci- 
o wd + | i $ sion region is a 
le / , ve New, C 


T 
Sf He cone of 3 radians. 
VA 7 v / 
re ae , rea, 
a 


(b) The distance from any signal vector to the boundary is d = Vé 


sin(7/16). Suppose x =(Veé,0) is the transmitted vector and ny and 


ny are the two noise components that are perpendicular to the two 


nearest boundries. Then an error occurs if the event {n, > d} 


{n, > d} occurs. Hence 
2- 


BoP iin 2d) in 


In general ny and n, are not independent. However we have the bounds, 


Prin, > d} <P, < Prin, > d} + Pein, > a} 


E 


or 
P< P< OF 


where 


rg 
Il 


> 
Prin, > d} 


a as sin (r/16)] 
0. 


is the probability that the noise forces the observation vector to 


lie in the shaded region above. 


(c) In problem 2.1 the average energy is & = 10a- or a = V&/10. 


The error probability is 


ll 
lo 
; ©) 
Oe 
[> 
“or 
“s 
1S, 
li. oe 
I 
Rte) 
O 
—_ 
A165 
rae 
a 
1S] 
4 
Sea 


Hence the signal set of Figure 2.12(a) is superior to the set of 


Figure 2.12(b). 
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2.9 (a) Before quantization the random variable y has conditional 


probability density 2 
(y-a,) 
a aa 
a 
p(y |a,) - e : k=1,2 


V To 


The transition probabilities to the quantized outputs b> bo» cues 


b, are given by 


8 
p(y|a5) p(y|a,) 
+ + | | yh 

-Vé, V& 0 8B 

#5 
P(bslad =f Gladdy k= 1,25 $= 1,2, 0, 8. 
Cle 
a 
where 


a, = 4a - ja is etet Coy ere 
B oe 5a - ja a = Pe Pe 
O, =- ©, By = © 


(b) Let x= (X) »Xy0+++sXy) be any sequence of N symbols from {a,,a,} 


and z = (2) sZoaeeesZy) be a sequence of N quantized outputs where 


Zz E{b, »bos+-+sbgh. Then the maximum likelihood (optimum) estimate 


29 


OF x & C= {X1 oXq2+%+ sy} is given by 


2x? 


api! 
= isa P, (z|x) 


N 
-1 
oo log HT Pez, | x) 


aes log P(z | x ) 

~ n=1 eee 
ay N 

min “25 y(z,/x,) 


> n=1 


where y(z|x) = - log P(z|x). 


2540: (a) Let uz) = ee 

ae 2 <p 

ine Jf (2) < £(2) forall z, we have 
Pr{z>0} = E{u(z)} < E{f(z)}. 


(b) Since u(z) aS ere 5° 2c fer all z 


Pr{z>0} < Ble}. op eae 


(c) Note that 


P (mn! ) Pr{z(y|x_) > 0} 


E teP2 (1x) } 


JA 


aS 


pz(y|x_) 


; a Py ty | 2) 


Py lags 2/Pyp(y Layo]? Py 2? 


»D 
y 
a 
y 
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p do 
=> Py (y |x") Pay x) [P20 3 


(d) Let pe -+ = 1 - 9 in (c) and we have 


Pamm') < Df Py yb) Pyle? 
7 


(e) For pe OO 31. 0 and x a: 11 (46 1 We have 
As mT es 2 
PCy lx) ae : y 
= 0 , elsewhere 
oO e 7 Nowy 
Bl) be ee 


where w(y) = # of "1" in y. 


~ 


The bound is minimized by maximizing (1 - () 


Then P,, (mm ) ap 


ee Ls 


only when x is transmitted and 


occurs with probability P,, (mm) =P 


The maximum likelihood 


. 1; otherwise choose xa 


coe 
N 


ye = 


~ 


The 


if Pp 
H = : then G = oe P] 
x ae 8 se 
Rieke 
A § ee & 
Pp = ==> Gs 
ie ye ee . 
jae age? | 
a m 


Hence the Chernoff bound is 


or letting 0 — oF 
rule is simply: choose 
Then an error occurs 


is the output which . 


Bhattacharyya bound is 


ys = 
10000 Tt 


O27 0017 01 


01:0 4:1 0 


O00 17 t 
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(b) Rows of Hw consist of all 2% possible binary sequences 
except the all zero sequence. The rows of yt having more than a 
single "1" form the rows of the matrix P. Then Ses [I, P] is the 
generator matrix. 

th T 
(c) Let h, be the i oe! of H Then. Wy © ve 3 
re ~ 5 eA L 
is a codeword if vi" = 0 or b> Le h : Since no two rows of 
ut are the same, v must have ie of at least 3. Since the sum 
of any two row vectors of ae form a non zero vector which must also 
be a row vector of He we see that there are 3 rows that sum 
(modulo-2) to O. Hence d = 3. 
i min 

PA WG ©. eee bo Ey is the energy per bit then for K bits there is a 
total of EK units of energy. oa ro ok binary symbols per code- 
word is used then each binary symbol has energy €, = E K/L. The 


BSC formed by hard quantizing an AWGN has transition probability 


AE) 


, ; 3 
iS 4am Grror eccure in the 4 symbol 


€ = 
ees a \,. if no error occurs in the Aas symbol 


Then E{e,} =p, E{e;} ae , and Var re =e p = p(1-p). 


Since ie} are independent for 
y: 
€ 
+ 
i=1 
we have 


L 
Ein} = )) E{e,} = 
i=l 


i 


and 


L 
Var {n} = >» Var fe,} = Lp(1-p) 
i=1 


(b) Keeping Ex fixed, as K increases B . = é K/L decreases. 


For small x we have from 


ee 
Q(x) -| ee 2 dt 
x V2T 


and 


The expansion approximation 


Q(x) = Q(0) + = OG) |e 
x=0 
or 
Q(x) 7% -—-+ x 
V2T 
Then 
Sak 8K 
b b 8 
E{n} = LQ )~ xf] )-4 
( NZ ( nN2% ‘: 
(@) 
and 
2&K 2&K 
“b 
Var {n} = LQ ° 1-Q YI 
( N =)| VN 


(c) (2.10.14) can be expressed as 


Pr{n = (din + 15/2); din odd 
Pp a 
6 = 
Prin. 2 da. /2) co. even 
min i 
or 
ce = Prin = d in’ 2} : ae odd or even. 


Using d in = L/2 for the binary orthogonal codes of (a) we have 


ie < Pr{n = L/4}. 


(d) The Chebyshev inequality gives 


Pr{|n - E{n}| = e} < Yar in} 
= 
Or for large K., 
Pr{|n - 3 ee 4 = = 5 (kee) 
(L/4) 
Hence 
Pr{n < L/4} = Pr{n - L/2 < - L/4} 
L L 
s Pr{|n - a 2 ra 
—(K-2) 
ay A X > 0 
Hence 
Pr{n => L/4} wer 1 
“ 
y Mags | (a) Lec x = N 
fe) 
a abe} 
Po) -f ao ape dy 
me a 2& 
1 s\2 
pes “(y+ < 
1 fo) 
a dy = P,(2 


P(t) 


{i223 
1 P| 
= “me: GU Ola + x) 
atx ae 
=e) 
0 Bs bd rae 
Pd) -f a a NG dy 
-a V2T 
26 ~2 
ay -&(y'+ ree 
3 ——~ 6 fe) dy 
0 V2T 
ae 
atx a oe 2 
= = e 7 du = e 
ee i x V2T 
2 
00 fo 
ie _ e du 
+x V27 
= Q(x) - Q(x + a) 
2& \2 
go “y- x) 
= Ps fe) dy 
0 T 
; [2E2 
es is 
fe eee 
~a Vin : 
Z 
aan ay i" af 
ce -— © du = i 
-a V2T7 x-a V2T 
2 


P (+2) 


2é 2 
fore) <a = sab aad 
: ag et No I 
a V2T 
ce 26 2 
: 1 “(y! wer =) 
j., se Oo. dy’ 
v2Tr 
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= 
2 


ae 
i, —e du = 1 - Q(x-a) 
co y2T 


(b) d= -1n2[vP | (2)P_ (-2) 4 WET (Ey 
= -1n2{N[1-Q(x-a)]Q(xta) + [Q(x-a)-Q (x) ] [Q(x)-Q (xta) ]} 
For E/N, = 2 we have x = 2. We find "a" to minimize 
f(a) = N[1-9(2-a)]Q(2+a) + N[Q(2-a)-Q(2)] [9(2)-9(2+a)] 
; df (a) 
by solving an 0.4 
2.14 (a) <= Since Xy> Xo» +» Xy are orthogonal the noise in the 2M 
Dea Plc fete oo 7a te Me? ote are independent. 
7 ae bar ane’ 2 
Since we have noncoherent detection let y; = IG of + ye) 
MO = i.°2. 3... MM, Assuming xy is sent then 
is 2 : 
P(y,|x,) = y, exp[-(yj/2) - (/N,)] + 1,(/2E7N, y,) 
and 


. 2 . 
PCy 1x;) = ¥. expCy_/2) m= 2, ..., M 


The maximum likelihood decision rule is simply: choose Xa that 


corresponding to the largest yon D2. Hence 


rg 
I 


=l1- Priy, >y, for all m= ee 


foe) 


al -f (vy, |x,) Prty <¥y for all m # 1|x, tay, 


0 
fore) Yy M-1 
(6) -36 
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© Yi M-1 
“f p(y, |x,) : - uf p(y, |x,)dy, tay, 


0 0 
(b) Here 


Y1 a0 -y/2 


i p(y, |x, dy, -f Yoe ayn 
0 0 


(c) Using the binomial expansion 


ya ee Ve 2 
-y,/2 -jy,/2 
fo - Lai (*5 7) i 
j=0 : 

a eat 2 
-y,/2 M-1 Tyee 

( a 1 ) 2 y see (* F ‘ a 

jel 


and the integral 


oo : 2 ye 
fo oonr 1, VIRTRy, ay, = shy of! GAD % 


j+1 
0 
we get 
ms M-1 i+] - &/(j+1)N 
pee pee oo C1): (";)are o 
1 
j=l 
Bd 
eho 5 Cie (*") 8/3 
2 
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2.15 (a) Define events A = ly 
m m 


IV 


m 


Then 
Prty. 2 y for some 
M 
*Prt- i A ix be 
Mt. 
m=2 
M — 
=1-]T Pria |x, } 
m=2 
M 
Certainly Pre Aci} < h 
ML 
m=2 
M 
Pri Ak ik = 
eee 
m=2 
= (M - 
Hence 


and 


1) e 


also 


M 
2 Pris |xy} 


2 


> Al 


2 M-1 2 
1 - ( rnd 2) < min a _ l)je ” f2 1] Ge 


ang for 0 4.5 = 7 


(b) Note that 


co 


—— 
IA 


IA 


2 ie 
se c - iby aed /2 a i} 


2 le) 
\e by ee 2 


Po i p(y, 1x,) Pr{y = 1 for some m # 1|x, tay, 


0 


0 


38 


00 _y2j2 \1 
-f p(y, |x,) 1- (i. - 2 (ty, 


oo 


~py?/2 
cf p(y, 1x,) (M - 1)Pe PYy dy 


d ii 
~  _(14p)y2/2 
= ™- 1)? oN, fe 1 (W2E7N_y,)ay 
0 ie) or 1 
- a 1? eo. €/ (140) No 


1+9 


lA 


ie) é O 
(M - 1)” exp j No (i) : 


(c) Although the signal phase is unknown, it is constant 
during the signaling time. As signaling time increases our know- 
ledge of the unknown phase increases and it can be estimated with 
increased accuracy. The maximum likelihood detector tmcorsorat ee 
this knowledge so that in the limit of large signal energy (and 
corresponding signal duration) the noncoherent performance 


approaches the coherent performance. 


2.16 (a) Add the first row of G to the second row to get the 


standard form 


PO 0-0 6 1 
Ot 6 6 4.64 
'= = 
G 60 60 1 8 1 1 Oo [I P] 
O20. 0 fi 
Hence a + LL ] 
P01 
P 11.6 
“ee = jena toes | 
as be eo 0 
tee eee § 
EO 1 


which is a Hamming code. 
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(b) s= yu! = (1 10) implies an error in the 3rd position 


since s is the third row of H. Hence'y = @ 11111 1) 


ik 


(c) = yH = (0 01) implies an error in the 7th position. 


Hence v = (100110 0) 


zn 


td) 4 = 3 for Hamming codes. See problem 2.11 (c). 


eau? (A)? TF &® (0). is sent then 45°, X,, are independent 
arma m i 


4 > eee 3 M 
Gaussian random variables where a has mean b and unit variance. 


All the other normalized observables have zero mean and unit 


variance. Hence 


Pr (erasure) Priv, << all im} 


M 
a Pr{i,) = OF 


Pr{\_ < 6} J] Prfrw' < 5} 
m!' #m 


Choo ol 


(b) By symmetry assume x is sent and Prf{correct decision} = 


Pri.) 4 for all m4, 2 > Ox | 
m m m m 


-f Prix <a for all m' z m, ae > 6|x 2A, = Bey es 


—OO 


-f Pr{Ayn! <a for all m' =z m|x tp, (a) da 
f m 
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7 ie es ts ee 
=f - ac Gl. ¢ do 
ie 
6 
ert oe et (2 g 
= —— - Q(x 
af 
6-b 
r 
2.18 Form observable y sh y(t)o(t)dt . Then without loss in 
0 
optimality we have 
H y=n 
x x,n ~ N(O,1) 
Hy y=xtn 
and 
2 2 
-y'/2 1 -y' /4 
(y) == a 3 ( ae C 
fe ae a 


Since y is a Gaussian random variable with zero mean and unit vari- 


ance if Hy and variance 2 if Hy: The decision region of Hy is 
given by 
ony ee) 
es 714 a 272 
sfy: = } 
h We 
=fy ; - < gn4} = {fy : -vend < y < v0n4} 
while A, - A, =f{y : y <-vgn4 and y > vgn4} 
Here we have error probabilities, 
25 = Prf{y « \ |H,} =1-Prfye« A, |H, } 
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1 - 2Priy > Vind |, } = 1 - 20(yin2) 


and 
PE = Prf{y e A, = 2Pr{y 2 vgn4|H I 
= 2Q(v%n4) 
Hence P,, = ia + P, = 145+ Q(vV2n4) - Q(Vkn2) 
fe) 1 
Fay) 
P, (y) 
t y 
_~— 
A 
fo) 
2.19 
Simplex Orthogonal 
(a) The signal set is a Simplex signal set with M = 3 and 
energy &. This is merely a translation of an orthogonal signal 
set of energy €' = 26. Hence we can use the error probability 


expression for orthogonal signals given by (2.11.1), to get the 


desired result. 
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(b) Since bo = 0 , there are only two signals of equal proba- 


bility. Hence the error probability is 


a xl 
ro = )= 0( (3% ) 


2 
since [eae se = 32. 
(c) Here we have two signals Xx and x, with priori probabili- 
ties g1 = + and Eo a Consider distances between these signals 


and the decision boundary, 


The point z between xX, and X> must satisfy 


B1P2 (21x) = E,P, (2x9) 


or 
2 2 
2 - xIl"- |lz - x, 11° = noen3 
yaa 3 
or d, ~ ds = N fn 
and also d, + d., - Il x,- x, || ae 3 
Thus 
j ae N£n3 EY or ee 
42 oe + 2 bee 
and N 
i ae e) 3é Oo 
PE Zs ON 6é © 3) + 2( LON oon 
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m 


9.20. tet n(t) = J) n,W (t) + a(t) 
k=1 
jh 
where ny, = f acer, (eat k= 1.2 2.4, a 
fe) 
and 


ne) = u(t) = yn V(t) . 
k=1 


Then we see that 


T 
fo com, eat = 0 Ke = 12... mt 


O 


E{n,n,} = io (ay, (B) da dB 


ff oe: BW, (oy), (B) da dB 
O70 


T 
ne ee: 
=fov, (a), (a) da 


oO 


and 


Also 
m 
E{n,n_(t)} me te (2 - Eo) 


m 


= E{n,n(t)} - ue E{n,n, }y, (t) 
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gi 
E} f cary, (adaaen(t)} ~ oF, (8) 


oO 


T Z 
= f 9(e,00v, (aaa = o,W, (t) 
Oo 


2 2 
= ov, (t) - o;W, (t) =U < 611 


m 

Hence the processes > 1, (t) and n(t) are independent of each 
k=1 : 

other and there is no loss in optimality by considering only the 


observables, 


- 
ef yCery, (edae 
0 
: VE. + ny if x, (t) is sent 
-vE;, + ny if x, (t) is sent . 


(14%) 
_ on 1 20° 
P_(yix,) = alll lie e k 
(y+, 
and P (y|x5) = TT L a or 
k=1 2710), 
Hence 


VE. m s, | 
- |; ge 
m VE 
- jy 2, oe 
= k=1 oO 
k 
(b) By symmetry P_ = P = Pp : For x,(t) sent we have 
E EJ E, - 
2. “k oe k=1 EG oy "k a 
k=] k k k 
N 


where 
2 ae 
E{N} = 0, E{N“} = aoe 
Hence 
ae ae, 
Pee 
E koe 1 
1 k=1 Oo 
k 
n € n &€ 
= Pr n<- + -o( [® %) 
k=l o, k=l Oo, 
2 oe os 
For Feary and & = oD; é. we have 


Ji). 


2.21 (a) The union of events bound for any signal set over the 


AWGN channel is (assuming xy is sent) 


> (- mn =] 
Eo Q 
ed VON, 
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But here 


2 a bei and ke#541 
D Comey . « 
Be ae €, k=Ft1 


Hence 


Lae) 
tH 
IA lA 
, By Y Ei 
= = 
I 1 
ke bo 
ey A 
ep Pe. 
a _—. 
pla ri 
52 se] z BS 
—— baa 
ee 
pe) 
Fem 
a 
S) 
—_———— 


which is the "Union of events" bound for an orthogonal code. 
(b) Since {ly 2 A for more than one m} and tly. < A for all 


m} are disjoint events 


Se Pet > A for more than one m} + Pr{ly | < A for alt wm}; 


Without loss in generality, suppose x, (t) is sent. Then 
- . M 
y, VE+n, and yo 2 2 
and 
Prt|y_| > A for more than one m} 
< Pry > A for some m # 1} 
M/2 M 1 
< = s ees = 
= Prily | = Ar s 7° 2Q\ ie A MP, 
m=2 fe) 
Also 


Pr{|y_| < A for all m} < Pr{|y, | < A} 


il 
H 
! 
S 
—— 
In 
> 
I 
8 
“— 


lA 
os 
| 
Pa, 
—_—_—_—™. 
219 
(o) 
“=~ 
> 
| 
oy 
wi 
~~ 
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Hence Pos Po SP : 
r zZ 
Next define events 
e ys bag ha all m 2 1} 


A, ly, < A-all n> bt 


e) 
eV 
i] 
rs 
S< 
x 
IV 


Ko 2, 3, Me 


Since for k + 1 


e,¢ ee 2A iS A} 
we have 
Prte, } < Pri{|y, | > A, y, < A} 
= Pr{ly, | > A} Prty, < A} = oe 
2 . 2 : M 
Since Il - x, ||" = 4€> || x x, | Be ee 
=~ +1 
2 
we see that the error event ey must be less probable than any 
other error event ey k 41 
Hence 
Prie,} < Prite } se 
and 
M/2 M/2 
Poy Ft ee Pri{e,} < MP,P, 
2 =] =] 
(ore) nt 
Ze 
eiee (ay fon) oma (EAE = ie 7 Sin Wot cos Wot dt 
os (n—5)T 
nT 
ey ae 
== sin 2w,t dt 
x 0 
(n-*5) T 
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=- Qugt cos wt 
(n—) * 
+ | 
=o 1eos 20), (n - 4)T - cos 2W nT 
0 
= 0 
Thus {, (t)} are orthonormal functions. The performance would then 


be the same as binary modulation of any set of orthonormal functions 
such as those for QPSK modulation given by Table 2.1 (b). Since 

o, (t) is merely the same as QPSK signals only staggered, the spec- 
trum is the same. It is the spectrum of the square wave f(t) 


shifted in frequency by + wW 


0° 
oe ) eee ) [xo bait) + Xo 41 Ponts (FJ 
nt 
2 
’ - Zz : 
Yon 2h Te sin (Wot + o)dt 
(n-1)T 
(n—*5) T 
2 : 
eae i cos wot sin(wot + o)dt 
(n-1)T 
nT 
2 : 
+ AES sin Wot sin (Wot + o>)dt 
(n-1)T 
nt 
2 
+ a cos Wot sin(wpt + >)dt 
(n—*s)T 
4 ps 
6 Xoo 4 sin > + X5, COs > + 35 Xo nt] sin 6 
' (n4%s)T Z 
Io y(t) [2 costwot + o)dt 
(ns) T 
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Note that 


(c) 


19 ME 


2 F 
= Xo, = JL sin Wot cos (Wot + >)dt 
n-4s5)T 


(n+45)T 
2 
+ So = cos Wot cos (Wt + o)dt 
n—45)T 
‘(n5)T 
vs : 
+ Xo (nt1) aie sin Wot cos (Wot + o)dt 
nT 
= _k i 
4X5, sin d + Xo 041 £08 ) 5 ay eh a o) 
, Ro oe 
van © in @ + ys 
Pope oe a Onl aed 
foe) 
fe, Po ngy (td dt 
nT 
af T le Ll rae 
= cos [-(t-(a a)t)]cos[—(t nt) ] 
(n—4) T 
sin Wet cos Wytdt 
nt 
T T 
os + {« cos ris 1s cos [-(2t-2ntst) J } 
(n-4)T 
e45 sin 2 ytdt 
aM 
a cos 7 (2t + 45t)] sin 2w.tdt 
+ sy 0 
(n-s)T 
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nt 


--if sin (27t/t) sin 2uytdt 


(n—s)T 
nt 


[cos (20, - 2n/t)t - cos (2W, + 2n/t)t]dt 
nT 


Yo oe 
n—s)T 


l sin (2w)-207/T) t sin (2u +20 /T)t 
25 {- (2u),-21/T) “) (20,+217/T) { 


(n-*g)T 
= 0 
Again the signals {, (te) } are orthonorm11 and binary modulation of 


orthonormal waveforms give the same performance over the AWGN 


channel. 


(d) For (nn — %s)t <= t < nit we have 


Xonoan St) + Xo p41 Poner 6) 


a 2 LD ee (a) ‘ 
= %5. yo 6° ae n - }s t) sin Wot 


2 T 
+ Xone] Vr C°8 E (t - nn] cos Wot 
But 
cos Ec - (n - 7)| = cos Ec - nT) +4 


- sin ee - on) 


Hence 
XonP ont) + Xon41 Songs (t): 


nm 2 T 
= Xo, Vr sin Tht - nT) sin Wot 


+ = eee 2 
2n+1 a 8 7 0 


a 


= v(t) cos [w pt +NV(t)] 


where 
v(t) |, - sin e-ne] J? + (Fo Feeos[z(t-no)]) 
= 2/vt 
and 
2 TT 
-X sin |—(t - nT) 
M(t) = oe aad : 
Xone FE 8 | Tht ~ at)| 
x 
= nee a tan E: - at)| 
Xont] : 
= + (m/T)(t - nT) 
= + (m/T)t ¢ nT 
Hence 
Xon?on t) + ¥on41 Panes ‘t) 
= (2/Vt) cos [Hot Sans tit = nt] 
g) 
= + (2/vT) cos (4 - n/t)e | 
(e) 


¥2 Gin (it/t) = = 1/2 = @ < 1/7 
f(t) = 


0 : otherwise 


has spectrum 
Z 


(2) a (wt/2) 
t we = a) 


=f) 
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Choglr 3 


34 (1,9) = -h z(Z aorpiyrs )- 

(o) ekigi= - [ Cqlip +ayp J+ CP apie | 
= -ln [ 42+ +449 Jpcrp) | 

By symemervy : i 1.74 MOXimi Zes . Hence 


Fol) max Fol.) « ln ae) 


Sine b, (1+ 2V0pC-p) 
(b) &(44) = - In Vip uavr+ ale) +4, FP * 
= -ln [p+ (4-205 U+p) | 
by Symmetry qt. e + maximizes Cond 
“s Fo(4,9) = - cn fe) = In2-ln (t+p 


€) £44) = -lmf (q49,0P ) +GNrP) | 


= ln [ 4-259, (VP) | 


A\the the, channel is not & etric y Fee ¢.) is symmetric 
in 4, amd 42 Hence 4,5 To = + MAXIM FES ES cE) 


max Eo(1,4) 2 -ln L1-4(1-5)] le (431 ) 
= ln 2 - ln (43 0p") 
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3.2 (a) 
Ww By sy wiakry 9, = (4/3, Viele ee Hence 
Fo(p) ein Z(Z4 4 poy)”. e oP) Yet)’ / 


=f ln 3 ZC Bele tn 3 
(it) By syncing 1 = (M, ; iy, Ys) » Hence 


F olp) )z bn z(Z4 4 Ply WP ) "tof L)"" (2 (2(5)**) ‘*] 


= ln 2 =>C= ye 2 
FF no 


(b) 
(i) ECR) = ox | pns-pe | = lw 3-R- C-K 


osps . 


(i) E(R)= _ [pln2- -pR| = m2-h =C-X 


33 (a) By symmetry Lis ae bag2 oc 


— He-ik Zia & yt PO Soe spel 
Eo (p= - bw 2(2 4 ply Ix J me ALS) 


Ao = ple nQ =>C= in Q 
E(R)= mer lpn -pR]zimn@-2=C-R 
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(b) By sywmetry se 5 tn 12 
Lek plyx) = pelB-p) 8x. Then 
Folp)= -be 2 zt 4 perertr 


=-(y 2(2 “(pl p) 5) | 
_-mofey"*[p% cone] * 
ha pin@ - -(up)n[(@- 4) p TF , p | 


Y {tp 7 4(Q- 4) pp 15 yp eB Plan 
EL(p)= bn Q - bnf(Q-4)97F perp ir “UPI 1) PRP pie] 
Eol(t)= 4 Q- 2m [0-1 Wp'e/P | 


C= E5(0) = ln Q +(Q-4) plnp + p bn p 


14P 


E(t) 


ay 
(4) ECA) ¢ 


L 
E, 


©) By Symmetry we have qe A = Ve and 152 1 - 
Consider (3.2. 21) 


Aly,4)= 2 qcx) poy ix) PF 


a [ (a1)"F 4 3 (- oF | + 9,(- 5) ally. 
We ik if conditions fer the optivium chotce of 2: 
(c) Lek = fe = f= 74 =O , Is = 9, = 1/2 


5 
ea): £68) = ( \¥ 


To check condition (3.2-23) Let for KM and ag 
4 ' ; 
x) PP co Typ 
. ply) p Cy ,4) = 2 (£) ay : (Lyf 
For X= % A2,Ag amd Gy we have 
Z ply? “(y, 4g) = [cou Co lP ayer Tip 


Chechiwa, for i where (3. 2.23) is sodi'sf ed Gives 
[cans sone (Lyi? y (+f 


or (.91)#F + 3 (04) Fp > (4) 


which is sckistied Loy Pp fo= ‘3. 
Hemce or mae % Ce ah be choice of f 4 cs 


= {= a: {ax 0 » s=%e= 42 | 
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(ii) Lek q1=%e718 = % = Ya, , %;=%=0 


O, 037 2b, — (siwilav connections From Ge, ds bom 
Oso b mot shown for | 
= "3 “(y,%) = A 91) cr + 3 Cod) dep 19 | 
vg et by 

For % 2 A1,G2,%3 aud a4 we howe 


otyteiF? oLly,o)e at] [ Ca¥ s ato] “f 


and for Xz Ag Ag 


Zeer wing = age (4) [canta scon hye 
For (3.2.23) to had we Vvequive 


i + 
*@) uO) [cont Ceo | "uF (91) Fp 4 36.04) #/ a 
oe : aS 
(A) > z Cat)? , acon iP 
~ \s {he te condition from thet in (t) oud thus is scdisfied 
for pk foah, 

Monet for pepo Ha ephimmn dive of ¢ 
sh aa: 9s =o = ¥, awd 4s = Jo = O 


We concdude thot Lor o<p? 1.3 we use the 
channel in Gé) fer for ECL) we we use the 
) 


chownel WA (v). ne te r £C WR Moy int or 


Of<ped we use only kh chawne| im (i) fo get om (b), 


$7 


E(2)= max | Folp,4)-pe| 


o<p<t 
= MAX {pnd - G+p) [sn +3Conig]-pe § 
ogp<i 


and 
Cz ln 4 + 30.1) beLos) +697) un 697) = 1.22 


Note: if P= Sl Chawged to a smaller value then 
cl possible amd loth chauncds im) and Gi) 
will be used te evaluate ECR) for R between O aud C. 


A shetd o¢ ECR) for cases (i) and(cé) is shown below, 


(2) 
88 J 


\u2J 


lw 2 39 1.22 


e d(y,4) 3 = qa) polx) iB 
58 


volily happews whew qx) >O. 
: me ye =k z(z Cie) ae ae 
olp.g) = be Z(Z qeopcyro ) 
= -ln ' X(y,9) F 


and thus : 
exp)- Folp.g)} = a ty) f 


For the waximiging value of ¢ let xly)e dly,9) and 
Ealpl= Eglg.g) thew for vcd ouhat® 2? 
4 


(bs) = (x) pCb: |x) 1” 
(by)= Zi qo plbjle) #7 


and es 


(EE 8 rere 
(bs) 2 fs ee T 
Poe ae 
( aX Cbs)| fe Ae en eae A ? { 
vs T . 4 
~ Ss Lael gt 
Now note thet 
a pt, 
7 ald? 2 x? : 


for cock k which is the vector a? . (ath)? oft) mulbroled 
by the Oh column of [pire | : rn 
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The ion (3.2.23) tren becomes 


af La | y wep }-Fo(¢) Uw (4) 
caw estuchity for y those Lowrponents where qc >O- 


or al ws (4) holds with equodit,. [ p Mi 
(b) 4, 7° «Couche squcire ee ett i 


ue [ P ars . a7 Fol) u 
eae [ eee] a xf 


of (e358) ond &=G,,..-,4s) 


Or 


A 
sou Le Sno £ cane 2*7 4 | 


}O 


Fae? 
Or 2 
ei ae 
| 0 tl Late 
ac 2.4) 4) [5-0 4) ¥) 
Cap 4) YP 


which is a funckion of p and bhus Varies with R. 


wince in (ata) oy 
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And for pe we apt 


q"| = 4 F om : | | 
pi i <\f2 - 4)* J2( 2-4) LYE 


() Ce Jim Eolp) = Mime Lo-Folo) . bine ale) 


‘on oo a 
poe lm [1+(ate -4)" 7. tn J1 + W]e Y. 


(3) E.(4)= ln [i+(v2-4)* | = ln (4.48) 
1 F(R) 


Lwl\.18 


Ss 


R 


wl) c 
yl. Cans 
3.6 (a) By symmetry gh (x) on te few oll x . 


pe 
aygye) (a)? = yebuvba 
(5) 


Y= Yart 
We check now condition (8.2.23) 
62 


% Byte) 7 aly, 4° = 4(aF +4) for all x 


Z Kg): 1(Q%e4 ) aud (3.2.23) holds with epuelity 
Y y 2 Site gx) 20 fer all x. 


(b) Eo(, )s ln 2: x6 pe -Go/q™ 
P . y (@Fet 1) fr Phy) 
elnt-In (407) and £U4)= n2- lu (149-4) 


EC de QTha = Lu Q : £54) > MQ 
r isa con. : 44Q, 


é = E(o)= LQ 
A 


F(p)-ph(p)  &@<R<€ 
E(2) e R= Kp) 1+*Q 


ee O 2 < 
14+Q°4 


ln G2 
1+Q 


We wow find E(R) explicitly 
R= Ea(p) =» in 14@F = lnQ =» QT. : 
140F R 1- bell 
4 
1-R/AnQ 


“Ele = Epp iii «be? - Wf ea) ) 


= la taJdn/ 4-2 \42 bw i; Ee 
5) ag 
= lw 2-H( Pn) fer Ro ZR<C 


3 


=> {:Q7. 


where Hlx)= - x lnx - G-x) ln U-x) 


E(e) 
ln 2 
4+Q"' 
_~ 
ae 
‘ Y 
| x v 
“a ea c. Ing R 


3.1(0)From problem 34 using (3.3.28) uth 92 Yo 
Ex(p.g)= -P be Lt-2% 9 Giz) 
Extp) = max Exlp,g) = -p be [4-$ (1-2 )] 
= -p ln [E442 ® )( 
(make. Hck G24, comes from — Expg)er max @ (+9,) 
— 14 S/2 ) 
where Zs z [poe ) py |g) 
for channel (a) (BSC) z= 2/pltp), for) (BEC) zp 
for (<)(z chawel) ze ip. : 
(b) E,(p)= ~ n(-taZ) zi luz 


2 (itz iP) 
Ee) - ? luz 
©) E, (a= Exlp) - p “BR = - rete 


64 


Q= -bo(442")s z¥lnz . E. P, 


[3 (A)- oo h(422), Zinz |) 
2 2 {tz 


4 


E,l0)= lim Eps az : 


eters = 


Ea) 
38 (a) By ype cheese lads = for allx . Then 


E, (p) = P in Z 4 z 7 (2 [reaper ”)" 


= wae a « Pin 
-: a(R) = sup [e lnrQ -pe = sl e nv Q- ~2) =a 
LS Jd ied 
/ 2G 
C= £0) 2 PB E(e) 
Lay (P) 
E J 


(b) fow Ci) , Choose 4=(ts, 42, 42) 
E,(p)= Ey (e, 5) = de ln Z 


Zz % $2 Z. [pcymipty) 7) 
= -pl L(a } é(L) Pe = -/ cn FL (442° )} 


Ey (1)= bn 3% 
©5 


y 1-¥, 4-4 
Ex (9) = in JL (442 MY a 5 Ez bn s- bez 


Now use (3.3.24) and (3.3.27) 


palo) = - 22 ia lm EZ /peyrn) pony f 


x x! 


x A( ints 68) 


ln 3 
z 


= 2ln2 
3 


are (n 3/, R 
for (ii) we first fry q0x= 44 for oll x Them 


Ex(¢, %, ) = “p ln ~ 2. i : (zipermpir)* 
= ph} [eager eco] = a bua 


EU). [w2 


E,()= ~ by (8 *) : A. We =} ln 2 
: bo 4 )- 285 Bae oe nz. 2 


12 p(se2 1-4) 


ft) Ga 0) ! 
c= i ln 2 ' 


= lw 4 
3 


Now we take, G2 tet ree ae = Ja = © - We hove then 


Ex (pe af bn rel = gee £14)- luz 


Kyx(2) = Swe pllu2-R) = © gc lu.2 


1<e<oo 
wz E. (2) 
“Be opkinum dishibution is Ni 
a.z( ifr 0, Ma ,0 ) a 


33 (a) lek 124, * Te = Yq ond check (3.2.23) 
a(y,% )= Zz 9 plyle ip - {12 Z plyin)itp 

o(b,) = 1. p)*P + pip |. a (bg) - oCby) 

= ply) dCy)® = ( 4] C-p)ip aa 


17 't ; 
5 A 0 hog [ley ap] Taylan i 
i (3.2-23) checks wh ow welts a. 
Ep(p) = -lu Z aly) itp = pln - dtp) ln [G-p)P + pe | 
ve 


Eo) < aa ln | vice +dp | 
EC) = lw 4- (n[G-pyFP +?p ia 


_ wplags)6- iF lu (1- 0) +e FF Iwo 


Cc: E.(0) - n4-Hl Pp) where Wb )e-x [nx -(-x) ln(-x) 
D)E,(p) = Exp, k) = -ply £ 22 (Z peop) 
Let Zs: 2 eCs-p) thew 

% 4 
E,(p) = =f lv A(taz r) i - tw [sz | 


E,.C4) = In| 4 |: Paes bn (142 Vpte-p) ) - ae pulp | 


KZ 


fal SLD 
“ “oe e Ay ZT 2. ln? 


44Z . 


Cer) = liw E¢e) > ln 2 


ees 
C= ln4-HCp) 
B40 (a) By the mean value treorem E.(p4) 
Eo( 44) = Foo" 4) for some p¥e (0) : 
Buk ECOG) > Fol4.G) for peed since Vt 
4 (e, V)¢o =? Eo (6.4) decreases with cs fe? 


6% 


Thus £(4,4) = foCp%%) 2 £, (4,9) 


(b) Toke vonpatholog ‘cod Chownel fer which 
lim Ele.) = Eto) cao tem 

‘ate: 

dime Exl ps4) = Eex (0) € Esplo)= live fo(p,¢) 

p-7@ a 

thew EC, ) = Eo(o, =O 

and \ : 4) 

Ex) = E.(4, $) 
both fumdhiows are Conver A and thus 


cawwol wherseck at more thaw two 
Poiwte so 


E..(P,4) 


A 


Bdkes 


Ex(e.9)z Eo (6,4) for o< pes 

kx (¢,9) ¢ Fol. 4) for 1< pec 
and simce the, fuonctions ave smooth £044) £544) 
3-44 (a) Ex(p,4) x 9 dae Zo GO) 4K) Hs 


oe 
thew ( ) = Cx = Ex( ) 
ts 
GSSummtiow Kyi i i nite Qx GL 
i have ud 20,07 defination e ff 
Ze [oaten 9) 20 


6 rie [ transformation G(x) = 6(Gl)) 
ad my daamolige Hi quodekt forbs to vr 
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F(4)= ZZ 4H) ey 40H) = Z( G00) % 


Where hy doys%q are the nonnegative ergemvilwes of X 1. 
But dase Be suw G QQ ae oe ee fous a 
so 1%) is @ sndtion of U frmction 4) . Mote that % 


is a@ linear a ue bepveag of 


VOW 7, pe nvexr tow 
Tevet. Coq) i cle Vim ge ad 


(b) Since ftq) = ZZ IO) YK) rex! sotisties Ae de. 
of the theorem iw F hependix 3B, applying (3B4) and (38.2) 


4 
a : Z qe pyleay poy i ¢ 
le * 
for oN 12a 


re 


vik oquclity if q7° - 
Halprying by qCu) \ and SUMS over oll k 
we BL qenqev) lyr = oy Ex Pes 


x x! 


Yherefore we hove ae 
diana 
slike plylx) ply 


with equals oo 70 avd where 9° is the 
“ohne i 
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3.42 (a) By G4. oh 
ag fe ne = ~£ tn 2Ve - Se wy) bn wly) dy 


where -£(- BE) 


7 +@,(ey) | a oe Se 
woe Leg AS »| wa AS 2 = e 


if we assume Es cc 1 we hove 


S Ss rt tial We A d'gen | 
_—— & Z@ +e 
am 


2 2 
Es 
o— -¥% 
Sere Oise ye : 
far le 


00 

: a oot -Y¥%, 2/4. 2&s 27 
vy) boydyz om (He (+S y? L&E (1- ~ | 

Sy pdyze® {se s1° ht Mo} 2 


“ 
eO-S)ie eo "(és 924 fey? _lyot fe y4 [yar 
( No Tae No >t “ai cary pr 2 fr | 


z(1- uw) [ = -i+% ~4 by amt - 2 & oS fe lua | 


fn Lb lw ate - Sa we wesledt clove (£5 ))" for m>A. 


* ECAWGN)E Ere aks for Sp, ced ax im(BA-8), 
71 


By (2 4 42 ) C(8sC) =v -H(p) where pefe fe : 


és 
(ee: Mts 1 ele 

polar Mane 5 fe = gfe 
y C(gso)# nz -AH(4-é) 
Now consider Hoc) 2 - x bex -G-x) Ge C4-x) and not thet 
Ui) = _lnux + Wa-x) , HG = -£ - rep 

Using Oo Taylor Series expemsion of Hex) around 4+ we gah 
HL Me-e)¥ Kl) + HC4) Ce) s WEIS... 
ln 2 + O(-<\-4€ 7 = 76" 


-w2 (2)% 


WR 


WE 


Cesc) # ( (2)és fr & cet as iu (3.4.20) 


(b) By (2.4.22) for ce 4 we hove 
E,lo)z E(o) # che Using the result im part (e): 


for BSC 


‘ 1Z 
Jas ee = 8 os cud lim —& we 2 
Wo ileal 
; s/No /No 7 Ey-20 eopy epy, zie 


oo m each case Gat 2 
—»0 4s 5m fo the AWGN and BSG, 


tsp) FE Mo 


By (3-440) Elo)2ln2-nClez) where | Eas oes &/ny 


aie AWGN : (im 200) z bv 2-Ante tn 2 zasc = Vapirp) 


és), 20 
yr BSC: bw Zo 3 Elo) = 
r aa époeo oe 


so Elo 
—> Oo as Es, 
[Wo ‘No 


Now for Ye extpuracded bound, b 

rie Y tases ie 

for AWGN Fex(o) = Es. => Eile) « yy for all €s/Ne 

for BSC , for lrge és : 

f= erfelbbe z oN ner lap 
[ep 


fine Feulo) = 4 Ge 
é hj E/Wo Z -L aT ex i 4 ae on, me 7 


13 


—>@ fe the AWEN and BSC. 


3.413 For shy << and by Problem 342 we have 


®Cy) = ur “(1-H fey oe ee (oly +] 


20 Ms 


tes Leto er Pg ye... ) 
Tyvering, (65. for W>4 wo have 
fo"y) = 2 (BE) aud 


P = 
RFS (ey) 


or 
Heo = =o) [ i+ e(xy) | > BQ) = wey)[ LEC, y 
where ‘fe 26 : 
wey) £_& gl* and eC). “ie fe ee 
yar 
Es YX 


3.44 (a) | et fo) Ae Agee. be the tection 


Probabilities ot the he two DHC's. Since the chanuels ore 
imate det the dvcusikiow bees \i4 of 4 he site 

probally of the, comp 
NG is Roy FY \z) given the tla Z) is sent. 
Therefore for { osite, cheadael Gk 


bole.) -e B(E, 9652) [RGmopeyias*) * 
where , zis a oval y os on the Oapud pairs. 


we, restrict 1 I Cz), where +), 96-9 ave arbit 
‘pul arab Oss re lawrll on aoc, aaa ) ther sat 


Eo(e.9) = ie zi (2 Z. 90) pl bore ) iG. Z ple) Plylz J) 


= fale) + Fo, Ce, 4) 
where Fy 60,9.) = ~ lw z(2 4,) eyo"? )? 


Jo, (¢,4,)z - ln z (24.0 7 ay )' My) 


_ 


Wt we hose qe (x) k. eon Eo, (P, i and #2) 


Marivwar E, \s OY Ge FWwen ten cb us 
| from Gay t) f [ , 
qx2) « = 7°) 952) and thus 


= max fy (p, )-e_ max E Cp, 
rides 9, PF, da Prd, ] 


BAS (2) Ne Wave ww dic oink chawnels and $ @ eath 
— disthbubion Jai © aud now We veld tye jth 
@(). Treating asa single channel the imput 


ek 4) us ec’) ee 6(4) oun - ++ 580 he - pl ie 


ows let (1) be He LMLLWA \d r ith 
Senne) nena = a "6 \ 7 


Figg) =-h 5/2 qe pops | - 


= bn) ol] 409 otyixP Js 


44 
i € Zils eT Zz ym a) J) asst? | re 


where X and Yo are Lhe ipa ok output ss ec of channel i. 


” Folp,8) = = Ln {perl oxp[-hy(p)] +... 4fstwiT “ep -B. p)]} 


Since on a ave dis swt meh ead C) ; é 1 mu 
a Channal v o t : . : 
Now 
Kole) = ~ bln f). Mar 


fe 


af Zfeusl i exp] Fe(p))f forge . 


Bul simee ¥o,(9) 20 fo pat os % hats 
cower U fundide im @ im Conse gut 1p boae oF atc 
conkitions ‘fo v ea te are Aven ul (38.4) Jord GB. a 


oy 4 wet 70 
ae on exe [- Fo, ol 2 with eelaf ny 
Broign fe Eo(o) ¢ ix will ly ‘ mn 20 

aud neve 

Z 60.) fights - Mie, inp Eo 
1p a 

a : Z (pw iin i expl- fol) J 


«FZ LPT oyol-e lf 


Haretore 


on veliky , wlide will bee yusbyied below ) « 
- mol) ond sla ee ee? ease 


cll 


[edui)'s Foutf) =v A | os =< Alea fe Cae ~ Fe: ©) fer call k 


16 


Yous ed) : eo [Eaxov/pl > oO , for a 
Z ealtawre | 


ond 
K oe) = -(u z Zieh op J- Fox =-l 2 Zofia 


ol 2H fh ly 


pre-e 
©) ste schetion to prolew 3.2 


17 


B.AG (a) Since the messages ™,,M2,.--,m, Gre L most 


Likely WRSSGILS thew we heave aS decision rule: when y 
is veceived we choose My Me,---) Mz, GS our list “f 


PCy 1X) > Blylew) for M Limthe list aud 


alt wit L 
op BO) Sg 
p= CY! am) 


So Ava Wet WA WS Fas +» aw error occur if Wk VECUVL 


x such vhak 


elnady LCP Sme) 5 |, for some yo alt 


Led poe! sm) wheve My £WA for 
it eae ® Tf ye Am | bm § 2 2 feel) 
= 7% 
(o) { ye Nw then e pC #1 xm) | > 4 for auy t70 
pee 0041 Sn) | 
Suwwi ovew all ible velues 0 at as +™M 
Lt) ve sud nalude‘Uoe ed | (me 
pea shvickly mow we aud Vere tore, the inequality 
is tll soxistied and tus ¥é Ki, where 


a 
nS =4y: Ze it feels) >l ) l>o 
mom gm [tel 2 Cy [Xm) 


Therefore 
%,. < Ze P (ylye) = Z pire) f OL) @ 
wheve fy (p is Ye indicetor fumdion given, by 
18 


Letina. y | = of in 
4 il a : ie J 1p) ©) 28 


G27 Za (x1) OCyIx Tag ee HK, 
yeY” xe% ate ma are i 


? 
AZZ A peylia yt} 
Since E(9°) < [e(4)]° for O<ps 4, we hove 


ot 2 aun Mae 
Th Fe Vera t fi Soe FO Th) 


2. TE Pa) P72 ‘ 


vi fv Zt : 
The set Aut WA wher =™ k 
see AH wi Fis 2 my =m for oll k | 
Since now all the weightings A(x Xu) ave eguon we spk 


i 


22.22 1) pA) (M4 y. Za os) prenainef ft 


ye” x eX” 
Finally since (+) Z (rH4-1)" , we heave for oes 


TZ rr (H- we 
ae ae qe peels M(H) Z , Fea peya 


Simca this ( de, same ‘or al m a ’ - 
tvivielly by a we mat nd wpperinuncng si 


sae nN 4 {+0 
? 4+5 did 
oa Zz. Z 1! pyle Ff 7 


where p= ft ard Eherefore o<psl . 


(J) This yields % < enp{-v[(p9).7e] F 
where R= ul os pel 


lower bound CSSUMID, L fimste and N—y@ is 


®% > exp . N) Esp(®)- of) | = eno affap(e)-ocw)] f 


where 4 
Q= Iw(HL) - nM ~bnlh » R-o&) 
N N Y 

where O(N)—PO as N—>a@ 


Esp() = sup [Folp) -pR | 


mar ae 


30 


E(2) = Max (Eo(p)-pe) 


ote L 


where 4o(0) = 7. Ho, 4) and so we'll have 


BAT Cowsiber 2 mess of lenath N aa... oO 
oad bb...b , thew foot (2-5-8) ws have 


ps) = N WJZ PCy) Rf , Ofs<i1 


yed 
a) BSC 
- bd p= N bn [py 08 tp) p*é [ 
po ew = W4z0 pes) sha s) 
b 


34 


(b) Z chanel pisd= N lu p> = Ms hup 


wa = : 
P i 
b _~<) mM 


1-9 lwp 


(c) Simee hn oe ie aes imped ave used, fhe channe| 


becomes 


a Ge pis): in] @-p )*p* 7 
ay, la-syinpssatep)] 


Se 
= eS pS), re : 
inp 


BAS Tale . be a conimmous random variable witl 
Limite mowew all order ( for a diserele vy, the 
reasoning is the same with suamatcous replacing integrals). 


© @i qe - Jr pryidy = fu u(4-0) ply)dy 
St 6) 


where u(4-6) = -f : Ye ries [ u(4- 8) 
| o 8 64<8 


Clearly uly - “o< @ s(¥-e fer szO . Therefore 


, 1470} < Se st ‘eeuse of «pep fe 
~ 


= E(e* t) ae E pert? 4,320 
82 


uk eo 2E [ a7] then Ms)e ln i; ot oandy 
Gus Riyrel V olns)-se] -00 
(b) to dokuim the ticles bouud we MIMI Ze the expo. 


newt , Ms) -39 


@ 
fls)= ln if a (4) dy ond lek p() 3 et pr) 


ont pope 
Shen 7 
Hay 2 LAE pod fs dy 4 
z =} Per ® ts 
Leto 
and 


Therefore, TMs) Cand ts fl<)-se) ¢ 
fuw iow iws.This cord alco Nea dale : 
ep) ood Ryde epeper inc) arc! 
vax Ha vesult of theorem (3.8.5) thet pls) = constant x) 
is cowex U in S, 
So we Can Minwnige by sdii 11s} -E =O 
Ts)-s Ms 
1 &{ el < et 3 where, 6 =Cs) 


(¢) It = Z a where the are independent and 
we \ 3 identically Pees ravdow yarichles 


? f(s) = ln E(e) z ln clea) = (n E(Tt (e5% ) 
$3 


dey — la T FE (e*™) Ec oes) 
where F(s) — >, E(e*™) - Aff? e * pon rdf all x 
a Bhyrat ¢ orho-swesr] 


B43 (0) v(s) z ie Elen) = | G-p)e® +pe” ‘| 


= (x (U-p) +02 | 
N¥%s) -_ Noe =O or rts)2 5 4 OW 


(i-p)+ pes 
og pe cis). (-p) —> pe" = pe 
. Se bn | Cop Ss 
fs 


Therefore “Y(s) « bale) (14 2.)|: bn [see] 
6(s)-s¥(s) = fe. - Fn | df ame 


= (4-5) nl4-p)+Slnp -G-S) let) Flu 
Then 2{ure} < od lets)-svts)} | ( art . v5 WH 
where HS) = ~§ lw - (4-5) ly C4-5) aie Sc My 


Note: ak $20 the oépowent is Bevo So we need s>Pp. 
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(b) T(s)= ln fe et as dy = Infe**f. s* 
Ws)2S= eo. and inrefore rG)- sfts)e xf --6! 


2 - 
, 270 


(cr) Shere- packing bound for problen 3.2 
() Fe(9) = p bn ie a E(k) s max p [l» $4 - e/: Co 


io) ie 
gitar for ol Rel Z 
A Feol®) - 
A 
E(R) 
lw3, R 
(ii) Fol) =pln2 . Esp(e) 20 for Gi eek 2 
lw 2 
Feo () = Ee) 
wy 
E(R) (see problem 3.2 and 3.8 ) 
eB 


Note : vm (Vv) ja pP) (Ss nowhere fight , while in Cc) cf 
is avery where tight | 


(b) Sphere packing, bound for problem 3.35 
3.30. Eole)=phQ => Exo (a) = oD fer all Belu QQ 
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(wQ F,( R)=E (eR) Ge overy where 
A ty! ) 


IwQ 
33b §E, ofp) = pr - (42) bw (Q-4) pi + pi] 


al Ee) ~ pwd -t4p) nf Gare + hp - ln (4) 


- PQ ~(ite) bw | Q+ igo “P nl ine eee! 
=¢ HQ - (4p) lw Q - bape ssa os ag"Ff 


QC) 
where eg ie en 
*- E(@) = -lnQ+@-1 ie p+ be P 
Q Q 
for 3-3c 
fal 20) 
BQ) ae 


wa 4.22 n2 4.22 


(c) Saw , Catena with ¥o(p) Os given iw Problem 3-5. 
86 


3) for problen, 3.6 wehoe £(e)= Eolp)- p¥o(p) 
Ks £7 
Here we can sdve i axphicaty . Lehing oe thew 
Esp(@)<lw2-H(R/lmQ) for all OC RK Cz na 
- 


awd (ima Esp (e) r iw 2 


gO 


3.24 (0) “ing inequality () im A veandix 3A, replacug x 
S , ue pave 
1 (Z ai)" ¢ Z ai® ad a (0,4 J 


Bili< z 


(0) aneraaing Bike aces of the m™ codeword 
Te T 
Bile) < 2 9,; Sm) z ft Yu Polen) Pte) / 


—— {het all codewords im & scohiefy {u0s)>0 > then 
Hag) men F 100( Z feqrowy)) f 


q(x)>0 
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Then we hove u 4 5 
Ble) <(n-4)[ v05,9)| < M [06s,9)] 


c ai aloe + vedity yes Yok aiven a code b=7h,-%ny 
ns wists at leet ann suck that the 


code Ou, = le ie ae er ene.” | satisfies 
BC em) < ee) < MS is(s,2) J Ms ,3€(04] 
Be ie oe cocoate tween be: Oero-9e.. 


(3) We com conshuct code Eves from men) CS ollows : 
consider code Cw)» f* aM codewords Lxcept Gee 


then as we did im part (C) there exists of least one 
Xu such that 

BE (6m) € Bi Cb) 
dnoose this xm One ea An vou aoe replaci 
Sons by £, and ee pina all the fe clende fw ae 
Them by (b) and (C) we have 


rN re is Ns ( ‘| 
%,, Gn) < BiG ace lss,9)} s€(0,4 


levee, w-1 | | 
B (Gu) 4 . Py Cpu) Ply om) 
w= 4 


. , — 

+ ZZ fab BO) 

© for code G2 1h, he, Bnd we hove 
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B (Gn) = Be 2; Vf Cy Kus) PB, Ll8mi) 
but 


B,(6,)< (8) 4 z Z ff plow) p pleut) 


m= whl 


wri 


= a Zz Ru Badgoysa)s 2 Zp Bn)? Cyl’) 


+z % faye pp bw’) 


wiser 


Aho Z Ewe 


for M242... 4M. 


2 


(f) Alber £406) « 4 2 B(4,) 


vi 
HM 
cLiBlé m) +h Zi ya ZR Xa), yPhu) 


M wel Ma) aloe 
Chasing tne Mies of <ummetion im the second term we hove 
®.(b,) < < * Z e mG: ores rth Z Al C115) py ple) 


mic, =| 


a 2 8 Con) +4 z 3 oe 


bectoe Qa) «8 ia (a) ed ing 
we qe % Cb) < 2n% Ly(s,9) | " 


$9 


(9) Yor any 9 we have oe 
22 oq Zipcy pyre) | < ¥(s,¢) 
We mow Find necessary conditions ow % ‘ec MiMiml BRS 


the | t hond side of the above imeguality . 
Let be ow eee mulkidier 3b 5 SR 


J(¢)= ZZ oq ra / PCy) pé ye) A a q&) 


x x' 7 
fren ne NKESsary condition iS 
ao S 
9003 429 ae & "60 7 (Pop y) = A206 
vik BUR" LD: Oh fo dap tr 


2.(6)< ar? } ae qooqen(: ; / eye) pF F Mp 


A 
O22 (G2 = 
( ) E TT ee ye ft Fle) 
rLZ 2 (Y Xm 
a RNG lu)" ae 
ae NG(m) © Ary 
where GgSoe) =] Xe a bn folie) _T(%,p)2¥ } 


or G(%w) 47 . PC 1 Xm) 2 ee Bet 


For ye Bu NG, (x we) we have 


a +¥ 


Pi Cxl%um) % "Bybee 


therefore 


ee 7 23 PCLaw) 
TA mt ye Nant Get) 


gr eater as) LF s  ( \%) 
i wz yefanGls') ¢ 


but Tn = [Fan Ge(t4)] U[An a Gem) | 


Lew 


Ze, , fy es a D, (phew) + ye By (lb ) 
Ye bm ye Ml GCs) yeru 6 (Kana) 


Since Kaw 0. GL eam) ra G(x kom) we have 
Z PG) g Zo Loplsw) 


7 


4D Gel Sw) VE GL) 
" Z. Oiled ¢ ZF BO) + Ze Poly) 
i a x6 Fan G, (a) xe ame : 
ew 


Zoe) » aaa. oe fe rl) 


Gop (Xanr) 


awd so 


22 2 66k. 
Ye we | He eee I (4) 


ie o . A © (y) 
wm Beh Z Ze lp) 


() 2 Roy). *} 1 bn BoCplens) - 3(6,9) >t 
er OCen a AS ALS) 


ve w ye 
aoe 7, In PUYnlXmm) _ TCP, p) — 
N mat PC Yu| Xun) 


Let ya . lw Pl Ynl Xnan) me 12... 
PC ynl Xmen) 
Thew Ye Zu are iid random vor robles wetk MLO 


Z = E{ zt = a Plyix) ln ply) « JCB p) ow, 


por 
By the weal Jew of large munbers , as > 00 for © >0 
O51 Zm-Z2t} 0 (2) 
N mei 


(c) the Comverse b, the Codim Vheovew S$: Lheve, axists “~>O 
such the 7 ae \ N , whenever R > Cz n2-HG) 


Therefore for large U ue have 


(ar a” LICE) +¥ 


since by @ We second term in (A) ac +, Zero OS 


N—> co, 


92 


= L(+%,)  ohere Gin Zplyls) and BZ piped 


Consider a “cha” BSC with F= 4/2. Then C= (n2-HG)=0, 
het te w(x ®X2) = number of places wheve Xin # Xan : 
Them decurly TE is the some if we consider wy! of x, 
avd Xe where they ore different. It is eguivelewt bo consider 


%:(00...-0) ua i 
nS 
6 (14..0. * 
wis Qt... 4) re 
y! 


bhew % - om = €, “ . on Py Cpl sz) 
Then as im problem 3.22 for ony tro let 


Ge ae 4 bn Pr (¥!l&) Tbsp) <x f 
By (yx) 


~ Zi Py lye) ze e MIG — adel 
a 


but Z , Gis) "22.0 
G4.) 


and by the converse to the codiag thoopeur there exists ADO 
ae ee 6. yw X for all wi”, aE ON bay Find 


33 


thet for P= 42, E20, w= - Then 
-w LICL pes 
> ie fe 


where bie): + hn Behl 4 se = — lm pce -p) 


Finally ao (m os } +0 | 


acon War 
aes 


3.24 For any >O Gnd awy xeX, Let 


ais 1 Cn fal xix) _ -Jpe rf 


fh GI) 
eee H 
ae 2 J pee > 1 12 J B plm)oy (1) 
ay llarn 
Asse asian ie) Pra cade hd P (ylxm) 
Buk Ye GCSu) impliee p CyIxXu) > fr, Gelem) 


Then by the same reasoning used in problem B.22 we hae 


Piyldidy > f P, ple) dy - f R.Cplea)dy 
A. Gein) Mw G (bu 
eS MPL SE Baily 

ae M mz Nant GC xm) 
. Ss ] § pen dy -( %, ef 
pa te (Sm) 
= EMPL 1% B02} 
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M=\ 


whee R= 2: 5 ( Rrcgeedey fs the error probeloility 
Mw 


whew code 6 \s used over Hoe, “dsm” AWGN chamel and 
Ble 2 F( Bi ytde) dy 
Ym! Gm) 
We now cont to show thet Gwe P (x) =O 
p-> © 
Woke thar 


; . Qlvlkw) . 3(¢ 
{ , (ylxw) J Fed le s > (Barr 
Elen t NC lx) : 


2 LZ PLY] tun) z ad 
Nmet pC pul Xun) 


Leb Zp 2 lw Pbl%nn) and as im problem 3.22 the fz 
( | Xww ) 

ove vid nd Sovicuee uit wean Z 2 EfZn} = Ip; pd. 

By the weak law oF log muUwers alate F>r0 


Liw ( Kprewddy ra live ae $ ; >x| =O 


yoo © (xm) V~>0o N mel 
for avery m= 4,2... oud so lim T(t)=O | 
N70 


By the COWWRYSE, To the codi theorem , there exists an a70 
<ach Haak fe ve fer R, > , Therefore fer large NM we hive 


2 -T{2 ICP p)+B' 
% y xe wLaesp eel le Pyp)+B 
where PIP?) = Esp(R) avd P satisfies G é Rr : 


$5 


Now we Compuile belveen cine fo dhemuels ywcomdly TCP; p) 


° © yw yu .. 
2 4 vbw) (we tate) 
:4 ( 0, Cpe { Iz ie iy- peek f dy 
ca pn pm ball by pel} dy 
a. Gp) I Xevll” = dp E 


M3(P;p) = Lp € = (1-py = ix 


= (1-p)" Cy j where by (2.5.3) Cr Fy Ete 


therefore Fep(e) = (4-[ Cy .(E a)" 


9.15 E,(p,4)= pod Z z goo gery Ep perspire] f 
let 7 de Z [perv pom) ieee any, % ¢ X’, For Ka xX we 


hoye Z pam z= ey , 
X= de, ee 
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Dakine 
or 
F.(e,g)= 22 goo ge) | Za PVH? f 
+ en qx) ax") zip pad ei |" 
E z qi + zy qe) Hx!) Z 4 
The Roan disteibaton is token tobe 9” ft, A] 


which is a fom iene choice since the ae @ 1 Soe 


are ot epee ont Aus, mo part ser gt 


nieee eeliee we ohn si 
eke 1 clear inch fhe signal should be iP cpcimbcle 


To cawry he mrexinnl ger iow (ex 
¥,,Ce) = max F.(p,¢) = 67.9) | Gagt then we have 


Fileal= 22 94x» [z Z poy) pyre’) 1° 


x x! 


% 
m yw ct Me 
E.b)= p de a SP lea az 4422 VO), 
tor Me het mM my cil 


ous 2. w(wr-1) zi = 2 , wet zh. sxoa-2 
we OM 


ve a yn MA 


(0) Tor coherent Chowne with hard uesdaat | decistow outputs 


Ne a, 41-? b, for . 12,3, 
Ge *  pbylas) = etek i" 
Og 


bs where 
A; Sy: Yerys al jet 
ba 


From Prockew L4 
pcbila:) = res \x:5 = “al 


2u 


2 "As oge(esfi8 


the probolilit 
ol le. Wen ‘aft § XX ts et eae : ( y 


Clear >the ita ocours 
het the Oi 4s chosen and , by te sy 
he ‘Madhem he : compte cheng a a oped ree 
Tao Us he Come f j#t ond Oye 
Thus 


Palem-4)q oF Pt 


: poej las) ae ‘ "2 in) 24,2). 
ne +f +#4 
Now vt compule fae 


7: Za phy) pty’) < Z [penla:) plbwlag) ah 
r (b: a5) pUilay) ie) bi lai) + z. Vp elas) pbulCx ) 


nf} 
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ates GEO one £0 


= —*_{(m-a)p+y4 pcs-P) (m-4) { 


b s\n echewent chawnel with hard S/O. out te: 
y problem 2.14 we, have am ~ . 


“Ey a HK 
es v@ Z, Ci) Yn Q@ Ae 
ple)?" *F 


Event ha is then the same of mw part () ond z is 
qwvew 62) | 1 
= -2) 4 p(4- e 

17s A loop sfapepord | 


where P is Ox given aloove « 
©) Coherent chawnel with unguerkied oops 


Zz f | plyle plyix) dy x' $x 
ee 
«fle of st) orl Be 


(me, - (y;-VE : @x as Xe fd 
a) vo t my Tee | r 
ion) Sd orf b fG Josh. s Ge 
xu & 25d 
“iit §¥) 


ie 9) 


“la, Lota ‘i { Seertate tree) 


ep} -L (4 - Sey re i) a si od 
(J) Noncohenent chonnel with unquontized oudpuks : 


Z= | pGMWRG) x'$xX 


Sf POplxi i) Pyles) dy. i Cy1X YP] i) dy, 


% {ipoer PO) s) dys 
whi 


met © 
When WEL, nts corcrare Bef rata = ¢ 


Therefore net fergereyle™ fy 4 an 
2 se )[ [ror ed +] 


Now we show ‘nck D=&,. (2) sckishios 
Relnm- 4Y(Dh) -(D/4) Ww (w-4) : where d= -wZ. 


4100 


fs 


D. Ey (2) = E,(p) -PEx(p) ; where e Scdisfies 


Re Eup = - br [dalmes ae fees aha (4] 


= (nm - bn [i nt) ] +L (maz ln z 
Fd tu-a)z¥P 


alum -[i-GedET folto maye%] 


i hued — bn [1 xin- 2 ]+(m-s)z? nz 
sym a)20 1 ;6u-1) 2/P 
Pat mat Gees (m4) 2% | zi 
[astm-4)z. eau uut) 2 gafu-i)2¥ 
-4) zl |b m4) Metz? | balms 
Hela Oo eee, | me 


= ly wm - He) ~ Jp bn (m4) 


where A 
Sp 4 -4)Z le aan HO) =-x« bx -(4-x) ly G-x) 
Law 4) 41 
But D: Eex (z) = E,(p)-pk 


= ae ee “Tp bm 
+e HC§) 4 ln (m-4) 
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4 (mag) 2 a bn 21 (4-5) lms) + H( ,) 


4+(m-1) z4/ 


x bn Spb Z FG 5) lu (oad) Fal Sp Cf) hab) 


- (1-8) ( Agel oe 


Note tak 
fp = (mz-4) zile — 44Gu-4) zi | 2 iP 
Gm-aN(s-f) Gor) z¥e (m-4) | 


thw 
“D . (4-8 Sin 27. Wz ef lw 2. 
f rf 
or Da - dbz = bed 2 fe = Dd 


Yhevefore 
2 elwm-YWl/d) (2/5) buona) 
3.26 Cnoosings massates equi protably we have 


(oe 4 >; 73 
t. ie we. went have, f By (p)Xna') = ae ace 
Ye Beles)” 2d; 3 fe some 
30 We Ke | xd) ip : 
z) |} 2b ee Pl hem) Sf iri 


: F  foCp lim) < Eg fo pam) © Z £9) B Yl) 
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P 4 5P 
(Yiu!) [440 
Z gy Zee = f i970 


. . F 
< ee dail wn Fo plu) | 5 P20 


ie Mod according to eon aba lity WLGSUNe, of the code, 
sdecion, ret vavt $ fh (yl Len) 3 fol yldm) Ag and 
[z, AS 5 oe \ ave ate wy Avera: a Ve OVQx- 


ey ensemlle of codes we cbtoin , for ofp <1 


aml 


% < . zz © (lel? (plead *P |Z, si 


<Z L 94) Pye) (41x) ty [On-D Z ges uis ]! 
Dekine Olx}y) 2 » Keg Po ond wole thot 


QAxiy)>0 ond ZQlziy)as and then 


in ale > yen? — 9,028) atx) isi x. 
sue Z qelfuey = BA ndependent of 


where 


q Gly) = Fey 9) (4) ( the chawve) is memoryless 
Z qo ey)? 


ond Qty) is Tobe the w-fold product o {xly). 


Rezlwm <0 ve hai avd thus 
m1 


aa - FC IYIwV) nN -»¢ F(p,¢,2,2)-Rp} 
% % Ag @ ra Ee =e 


w F( *\ & ld Z a(x)? x 
ei ayy ? a ry 
amd 4Cxly) is veloted, 40 Pp! X) through W, 


3.27 The conditional prokcbi lities for this Channe| are 


s 


qivew by | ih 


5 #4 
3 2 


The decision vule icthe Same ac for the noncoherenit 
channel since the same decoder is optioum Vue to 
Symnciey WE CSSUML without bss of generality P bot %; 
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The prabockility of a corveck decision qiven %, is seul ix 
thew 
Ye, aRiyr Yow m ; meal § 


= { peunnL te, § ("on \%) Syn | dy, 


4 : ~2h, 
For “eh, [pe t)Sy,, » fs a cen 


c 4 or bho clue of er is the Some 0s 
cua 3. 28 (b) MPs rig 


Z: {m-2) 9 + V40(4- ) (w-3)_ 
wd 


For dhe Case of unguarctiged output vectors we have 
04% Pralxd pug) dy #4 
rT Sir Plyes) Plylxs) dy; 
mn [i PY! Pla 


et 
4 4105 


Coy MEd, ue the last product is wi therefore 
z=[ | Ra POAT 4y J 


90 4 
-} ee exp | -¥ (se, (f ey 


at . ? oxp]-# (aft) | 


3.28 k,, 2) = Ex (p) - pF, (p) = EC) [ee 


y lime E; (2) R<kx@) 
fe peas «P) <R< @ 


By Synmely q¥x) = VQ § or on\ er awd 
Ex(p) = -p Iv pa Z gx) FD f Z jplytx) pve | ie 
=-P ly rz exp f - fdosoheghg § ot 


x x’ Q? 
ond {hen J $ -di/ 
= = ( & g malp a di . f 
E.(p) W eee 2) P Pp # Sah 
77 
hen Fie Fy shi 
Vv: a). Wide ysl 
fl) = gp Soh 


Qe Ele) s -4D- 4 Fo. sD. bf esi 
p Q ist Q iz 


and for {<o<@ we have mara [-10] . 


For the siqual sek of fia. 2.AMb we hove ey iy-x? 


Haye) = bm (LVpcqangyn) dps bff oF dy 


ln ( ( = aie Ny 40 asl me of wer? dy 


oe Weer? — x'1\2 
ad ad eo , 


Ix-<l = ette dei’ (x, %2 24 (1-0) AF 
thew 


dy AG cox LF) kK = 42... 46 
2Wo 


3.29 (a.) The optimum decision regions are 
hn. =} : PCy) xm) > PCy Su’) , adl mtn f 


u 
ts 
E 
zs 
4c 
x 
SS 
he 
=.) 
a 
_ 
en 
~~ 


RQ, < x (2 acquit)? oS Z a,Cyinu) 


Re <u fz z(Z GW) Ply }FP 7)? (" 
-N{E(¢,4) -pRf 


ane where 


= ene 
ond wheve and eek act for oes 


Fole, q) = - W}2(2 40%) o 0) Yiry 0) Pf 
(3) ies (4) 20 ie ae 
eee eT = I(); ed) co 


“teas 0 2fo o(¢,4) 2 P70 
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and 


wo a 
Ov al o > £6 P.4) for po 
Choose, ¢, such thet 1C4) = Cc. 
co E>C we oe es for 
some 9 € 14,0) we hove 


Eol(p,g)-pR 7O ons Clair 
Kee (2) = mca: (Eolp,4)- ) >O 


“Az <0 


@) kis vesult tMs Us Seve. cact codes es el t 
Of covreck decoding, aS scdlisfies as yA 


Oe oe with Excle) 70 For 25c . 
There ave many bad cobs we cam think of thet 


do worse. 

What we are whovested im is Boye vere for all codes. 

3.30 Or -f b, Cleae S 
: a ra ly by (ae oy 
Mir nr SE he 

a for kz 1,2,...,K 

ay ip a Ox, let p= i-P 

Fo(o)= mox (o,4) = fole,g*) = a 31 F aay) oCbala ip si 

ae ie ee ; raul pela ) 


=17 z LE (er 7 a =~ Kf L ping]? 
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:; E(¢) a pln ke <(44p) by | pis ? i | 


a be 
Folo) = Ink +_4 orn psp ela B - buf ai Epa ge Ei 
21° aad fa ‘i 


lt $4 phe, pty meat yg 


” Klee ek +L [Ska p44 -s)ug ] - ln] pip +3 | 


=k + She pe A! -5)lu P 5-5 46% | 
-(i- $) bn (- Cura 
= ln K + §lwS 4-5) (is) 
= nk - WO) 


(\) C= ae = [lnk - Hey = in k- Hep) 


co 
©) a Calo e £ sucht fe | 
ms For exch of tere oe i rk yh laced 

¥ im Yu Therefore ee as | | 


thet Cow esult In But the 2’ N \ 
eee. “Thus a ae posctle distinc 


(d) Lek Ey = Xx eft . k =23,.-,14 
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BAU Se < Zz & fxue ff (Hie HZ)" 

Eo(e) = ax: te ln (FF =p vk "4 

E(R)= wax TE(p)- gf = max [plo pe fs bk 
Os 2 


am & © = 
in BO) we opt 
= y Qn. -N en -Nink 
ee ee 
: a Nim eS 


(9) Sov pee , Fol) - ol K 
ECR) = mar {pink -p2 a Kee. 
for VeC;: Pea 6 ( moiseless dnawnel ) 


AZ Suppo Xi | is sent ond dative, ave 


Bal 


Ei Af ily. x I< lly- xl f, where y is He received vedo 


dit 


Te Pskily Of eter Gun that, wos ent Sram: 


ae 2RiY OE {<z RIES 3 fore oko tindialy. 
_ IX- wll? 
2 {Et mics 163) ce 4M (using talkin) 
No 
xe - x f]* 
“ey £2 oxp f - a Gud Now we Onernge 
the bound over the ensemble of codewords 


os (Xk -%4)? 
~ Axe -x. 1% Nv : 
tar <(M-) oO awe er 2 '* 


J || 
Kile - Xk)” - 
ewe - Bose ena 


Whore Cy 4 log eh =" i 


Gleb za 
= tk -— Mk a, a ae ae. & go-ussian, vando 
Vaviable wath avo Meaw a 
Thee Vawiontg, C,? QE. 


Co = -log (41407) Ee (og 2-log (112%) 


3.32. , 
| Plyix) = LC; dyx ) 

2 Se ro Heaeel 

s i , yt 


3 

(0) ‘By sy wrmeiy q(x) = VY for *=0,4,2,3 

C2 ZZ myx) ex) ln ply) - ZZ poynle%ln 
enh ae “ poy bl9 oi 


x#yY 


ra [Tye ils Fs 4/s = nv 4% 
3 4 4/4 


(Fol = ot Fog) = a(p.g%) bz) 4 remote] 
+ A+¢ 149 
oly “face = -h4/ayfe eld 


Cc = -PR |= w4- =\w 
FC) won Ww 4-2 5 Re Colvd 


Belgas -p hfzZZ tops Zp GMP IS | | 
= np efdott [4524] } = ph] (14 (By)? 
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Fya(@)= max LE..(9.9% -p?) 


~ _(udilis 3/2. >%? (EE Ue )CH 
bleep ef anba 


‘& 


Paso io Bip lie ol tunaye| [0 


E, (0) = - = 2 qoOgte) (n nf Peer poy 


Z 
= i) x 32 lnfoat = = mS. 


Rep (0) = Max {Felp)-eR J : morte CC-P)F = 0 ; for REC 


ee: ? 2O 
(nd 
: j FspC) 
A 
B(¢) 


R 


Cel 
3 


2.33 (0) Consider the binary codes of 2% tad 
dimensionality Vv -(K- 4) po sed . This ary pga 
limeay Simce our Origine code wos bimear. Mow dew [Y/ dg. : 


ax the minimum weight vector in this 24 code. Thew 
Clearly fe RA < bin (the chesen, Cock is a Subset of the origi | 
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Lwiewt.. Std! forme. boxy code: 06: 20; coderecevs 6 
“haan Ol Kj j 1 we ve i 


ociy 2 Me <4 (24) or dmin g N=lKZ) 2) 
ti) )-(K-4) 
(b) We kwow thet Ne U and Onin ¢ , thus 


c— N) — ore € we chose OSH oA 
ares wags at = enter a and thew hey, 


Ae log, W - ~ 4) a Lt AeAck , by pavt (0) we howe 


_ g N-Klogn-% ia 
dyin, € Sekerlegy =n Ne) (1 


ywow Coit 2 on or K= NE 
M 


in 2 


Sulostituhing for KX and dividing bath sides of (4) by W: 
wien <{4 ez nef N \ 
'S) 


N 2 nz N) 2% 


for N sufficiently large we hove 


dws (oak oft where o(-L )N22 
ei a ie C2 
(©) the only veevichion rears ier lean ok ss he cde, 

army bine) code ois Tue dunn € daca (some tubset of the code), 
he cowsider the er subset Of 28 colevecty vith rsh 4 


wel te avo. Them dyin (binary code.) ¢ doin, ( Special] subsel). 
hove the ¢ iv from Cie al vecbrs of Ws subsel bien 
Sania. (4 fie ED aat nebare with os po ro ren-oved ) | 


this is se fe amy K, J sO we are 
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3.34 (0) Choosing @ bi vector comses the removal oc at 
— £0) wie inal the list. % 


So other ra: H Codewords if 
1 Z(") <a 


Yen if Is Be | possible b dioose another code yey . Thias, the 


emededs mincles When 
"At F( t) > 2 
v=0 


N) 


or 


My 2" 

d-1 

z, (%) 

v=0 v 
(v) From, proven 3.49) for Y= Zy, , where ¥,2 ord 
oa pvekcki lites Ct-p) and p respect pind y we hed 
N-O 6 VKCEW) 
RIX >r04 « (14) ? @ 
but 


Bi veide lo gf Mente 
Burd} +2 (en pe (tp) Ae 


Note Haak laa Has fon asec 
Hip e « Lain 
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d 
Then 5 Z(¥\G- py ps ep pt oh) 


ond for P= Mf We agh 


ES 2 mind wt (duin/w) 
» El) eo > Z(t) 


2, let $c4h scckisfy H(S)= lu 2-2. 
9 iy schiefying doasst Fg Sin 


Them rom (A) (0) we mee a Code oF minimum distance 


ee ee 
() From i x (3.7.48) 
~duin lv Z 2 -Sluz 
N 
for Ze 4 new 
duim y § 
; Vf 
A é ib 
Wnewe Beluz -HS) and $= PL; < i 


Se 
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Chapler 4 
AL 22 fo sie States:fo,4} 


4/44 
scl Se ho 
O/o3 
00 
00 
it 
00 u = 
16 
ob 
00 u = 7 
‘ oN 
0 
00 
| ul = . 
4 “ = ol 
1o 
00 
10 ol 
to q! 


tvellis dicgriom 
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2 K=3;RaM/3 . States: [00,0/, 10,11 


IG 


fig 4.2 of the text we have the 


3/4 . In 


VICRS + 


~o=- 
= 001 : 
Y 
G= 010 Wi; e 
0.9/ 
Ye cy) Y 
SP) 
dzorl VA WANA ; 
MPP 
AY 
22100 4 : 
Nv i 
YA 
‘1h 4 
frtOl ¢ Alli dh ‘ 
| 
YS 
va M0 « Lp ° 
Fett ’ ® 


Tralis diagram, (partial less clutter - 
frow each ode leaye sige twer’ connecting it to the ¥ 
voles inthe wxt step) 


/2l 


(2) 


[22 


Ij Cees aX 


Lower pork chosen at {; ex 


J23 


ibs D4, + DE, +DE,4D° (5) 
Fee VEE. +DEq4 D Q) 


Fae GeV Hr gt (3) 


T(D) = De, 17 F+D°h, -D'(aththe) 
Fram WH2): % = D-DF +(D3-D+1) pLzpepe 
From @ m8): Ee =(DeP*-D) ye -D 4 oor 

D*4D-4 
From (4) « PAO TES A 4) Dd" %™ Nyt Ye O 5, 


thew T(p)=D>| Dypt-2D-L _D | -D* 4+D420-D*p4p° 
D4 -pt4ap-4 4-304 9-0 


and the winiwuw dactone is Jest as. 
52 3 4 5 
() © y » 6) Medea - 047404... 
ans the winwwuw eee amy a\\ peths is 3, 
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[S=Dst and TCD) 2 D*Fe -@) 


Bex Vo +h (2) Solving G)-(7) we as 
¥4 = er OF, (3) “6 p44 p>- on 
Jie Piette Tes 1-29-93 

% = DE.+D 7 (6) Gud from (7) we 

79: 94sPhH, © stp) 2 p& (44 -D*) 
in = D%,+D%q (1) i-1D-v- 


ond the winvmuw ay is deer = 6, 


47 for all codes we howe 
Re TC9)| od Rx ton 
i: 
D 


ar Zz 
=z I=i 
@) V« TCo) | - z3 44242Z%.23.24, 75 
D=Z {-3Z24+Z 7-24 
Ge 20a) 9 ps(I+D12 4228 471 -D*1- PAT? 


at [pve "QL 4-2DI -DIF 4 D512 - DIT 
{25 


"pg zt daze 32z2~525_524,475,2&27 
(4-32 423- z4)2 


(b) 22 a < 2_ Det 
gra b 91 4-dI 


=a z* 
paz 
rss (4-Z)* 


(See 2a 


4-22 -z$ 
6 
ar . D°r I+0-D* ) - 2° _2-Z-227424,25 
{-201-D3 0:2 ( 4-22-23)? 


I=4 
ay /) 


48 Q(x)# erfc(x) = “Le doe 
ee 


Ub ead fe show that QUEP) < QUE, xro;p0 


ss x 
Since x,7 20 => fety “1X70 => Pe soe Sige s 8%. 


JAC 


43 be up x,G be the doka(rou)vector, code Crow) 
vector and dpnew mnoorix respectively , sud that 


x, = WiG 
and sari the code in = form with Xi‘ $ as 


oe 


on 


a 


% 


fi 


[x] 


ae 
L32*| 
(6) Conciaer ita | 


ros” deieraa 
a” 
fe) Jo Sar 
ji column of 


“yr * 
ite Fl 
Abu fi corpealng he 
: cant 


O10. 


O00. 


00.. 


coluwn 
case every 


Colum. 
theve is 


ig td (Ss4 ve 


ra fe] will be 


2 are aa 


e oe @ 


t1 00)... 
os os + mars 
O10... 


7-7" rf &e w= 


Wz 


aflh Cither it consists of 
»e oe 


eee” or ei 
bal one, 2 mw -p a 
he ales ee 
fag sadly ty Coluwan 161% 
lg] ws whl be de limece coutbinat 
cohomos £0] 
hee tal. tthhes all 


(8) 


- =. 


- 
w rr re oer Cc 
- _— => -~ ~ ese = © 


-—=lUWw 


- Ls 


ee © @ €°O @'' 8 ©: « 6 0:8 


2s will a 
ak pong one 


Ot cohtyae 


— tombinabibns 
utnt&es as rows, Then, “ty looks th 


As 1X Coan be seam, hes "lowed | d, oy ees ratte 


m the top, 25" © ¢ wed byZt s , follovacs 
mon. by 24 rove SL pepe “yfarefore’ by, ea 


rar go we have Bon ak aga ae fai 
: J ube ¢ “peo pare re “or 

it will he all hg . 

(b) Consider [x1< as w (a), the wodiris of all possible 
codex of th N Nee Cae the all code, 

pare t cole vedlors left Uh engw 


from (0) Also i ore comm in(X] is ecther all gros” 
Hy bea Cin any order). Assume now 


[<1 does wot have. a Ss Column . Then the 
‘oues” ois will be 2k -4 and 


so, the creroae umber of ‘ones in each Code vector 
wt k-4 
W (bjs N_2 whee ke B-Ktl or Bz kKek-4, 


and where B ix the wuwber of brawdhes for codes wel, 
and iw caddy branch we have Piaget per 

so! N= Bw z(Krk- i). EE there exist s iy ave 

with all zeros. Uae sat gs au upper bovend on wf Ck) 


Voy Ck) ¢ a _(icsk-H 
ak = 


(C) The distante batween ro codevector and ak. zero veclor i is 
senha! of ones”). Fromc(b) there et Q 


to its weaqht (numbe- 
mer whose reat less ow shay to Uby Ck). So the minus 
have a free distance Hal ik d ok “af, 


Ratt vouries from Lup so dr. € oo y ene 
2°. 
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4.10 Let % aud ke be any tue codewords with d= w(x,X), 
Varine decision YEGOWs 
Ay fx: (yOu) < wly ©x2)} 
| Aa afy: w(x Oxe) € o(ZOX) $ 7 

and U NEANA, = fy: Bi) = WCLOXe) fF Uther 
pinnae i Aa = ty Lge of ars iF Tp we 
count Ve AN, AAz as on error (whide will be trie hale 
of the ime only) WL Upper bound the Probabi ty Of evyor: 


A ¢ 2 2 §y (Pye) Z Us = jolerr df 


where © is the bi ervor vector. Vor d= DA we have 
Fr 4 ule) 3h 3 =% Lule) 245 while for de 2-1 we 


hove BI v(2dz dfr 2% [vlg)zt-ip a Rfwegjr€o 
becamse 4 com only assume, i - Volues. Thus we 
have thet Fe {w(e) >» dz 3 is the Same for dz w(<@ 42) 


odd or Weu, The Bhattacharyya bound begins with this 
Yound so we have , whew J's 


7; p ott 
fvow (4.4.5) we have 
(4) <2 ald) f4 < Fi ad) zy a eoly ee 


=dp deven d odd 
vote Hct 3 @ . 24 
5. atd)Z 2142 as) z4 4 Zac (-25 f 
d evew 2 did dade 
¥ ot 762) +4 11-2) 
amd 2 . 


3 F atzt- F ach (-zj4 
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‘ fet y ae.) i 4(Z T(Z)-ZTC z)) 


ond 
RG) £1 ; (442) T(Z) +G-Z) 72) ¢ 
2 : 
4 
a res fe Us, Us,... be a bi vente and devole 


\yront as Lz) 2U44u,24U,2%.--- 
3 ee: el tonal Code i$ Said to be cofucle 


J w i mpul seq with meinitel 
mr vo re dg pase’ td Gy the crrespading on - oud! kas? 


vat Ta). 4k Bes jynomial 
= di. mod o ny Elon uowma 


corres yuomtal Ic 
Fabs, A(z) rey WCZ) where e 2 


, hloy FY O. Thew , the i 
go wings - TG) oe Cabal retotes a 
mo 


igiks and the out eee 


waw 
i 4; (2) Ste) be d,...¥ Pat be of fimi rv 
aud hewce te corves sequewte will have 


hf wow wougyod dot and by Ve sinits "ow the 
shy hic. oe She Ss we havea 
bh ri conveaiong oe efi ee the 

a H85 Ww On the reatest LOWWMON divider ae Violated 


ie ged (42), crn ™ where m0 


Then we bnow thove Me mice of auste vee 
ee Ones o- ae Cz) cil peg Tf 7 


fz) Gq (2) + B62) 42(Z)+- s of f.2I4, Cz) -z™ 
Multiplying by 1(2) avd ay T;(Z) = qi (z) £(z), ali 


/30 


We will have 
pine (2) T@-. *P (2) Im (2) 3 Z™ TC2) 


Simce Ne code s GSsSUuw cotoskrop hae 
some Cz) vith i Tt ca sop WOWZEYO ag t¢ rae 
should itt: T, (2, bz, nm iwith Limi tely 
NOVUBYO hak v inn ne lage side of above vue 
will be @ pAnomial wth hone cfe deag and ri 
side vill hdve infe imelely man digits , which 1s 
a coutvadictinn. U Therefore a cai rope Node code, should 


have itsn G Common 


faclov A(z) ra of dear oh ys atop bres SOT ES, blo) £ 0. 


412 
Consider ae binawy sia code with vole Vm 
cud courtraint K, with code geneva olymowials 


i> Go. * 4s, ae Ws 
42(2) = V2 493,224 °F Fyy 2 
a (2) say eens thoy a 43,20 

id bey gcasty _ Should be nom pero . Also, 


K-4 


githoul loss a ol - with @ pourerc 
covstout Pinay ie re | ie LL ise a! 
l Fes a : Bes | 
$92 ay vies "Gud, 2 


| Jon gia os hod 


where the demaits of t i column @ all Vo 
ot least owe ea sates) ari rane Wee also nee Then 
theve ave 
Nk oe. 1) 
such a ig or Cfus MK) convolutional codes with 
toustraint K, sinta 5 aes —) 2-1 possibilities fer the last 
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=a and 2” ever ihhes fo v cokmmus bid 
‘ ony oue hilt v" pence 1. Uk know = del 
aa wl! Bez) i te greabest gs ie thet Z de, 
| 2) and he en Polynomial 5 ite wunmum 
7 the delay s ane 5 fi the st. We consider only 
sa poly Tet cm the same cklay or = members . 
ed onl ly tad. of the w pal ouials af a code. 


Ty we 
aw (Z) ree We thew, the dela CZ) 
2 ge z Kare ull be pie Te code & hoa 


ZZ) us greater thar O Bona 
ol ad be — the above > Tie ee of 


K-1 avd ual fo the numbe 
hg sgt ee yore Ar win , whic is 


WA 
— NV (x-L). Therefore the Muiuver of sets of 
1 ud arbi s AK — 
-1 G pier fone fo } Bute 


ot our Previous result we 
aie = 2 Fear “a” -4) = SS cate 


(ie-l) 


het TK) tee codes 0 ree K-41 such 
Prat the greatest price oe O f 9 Generedor * Bt 


manoalk Us ts wich isthe, sawe as Pagal eh 


Hak are wot cakastvophic Toa pale of 
yee -4 - arbi}, sy ey y be w w & 
UNIGus Way as ° vee K~-w-4 
cna 4 { Lowwow divisor a J and 3 ng va pobyvipmicd 
of chogren. i wm - Theyefore 
Ak) = 2, CCK) A(m) (*) 


W=O 


bu Atm) , woanbor WOUUe \wnowioks valwe. iba 
So frow tho paiaio wh hy sear she mcthod 
We Cow nasily find Sak ri: gle wae we 


J5R 


Equation | (#) looks like disereke cowdution , ther Sache 
ff Nix) is the produck of the transforms of PCW) dows 
calastvop hic aD is Hen MK)- PUK) and io 

"y Cale sivop ic Codes wrthe ersewble 
pee ey mel { couddaivl K is qwven by the tractor 


€(2) = NOK) - a geen). ahs rag ey 


NK) hie ag Oe =) "4 
which is independent of Ki 
As3 
ge ee zt 
$01 | an D- et, , 
i? iy | )- ts 
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AG 
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Q) We vote thot when a poth diverges from the all-aro 
ay some uode ; avd rons ‘aon dy hin beat time A ee 
mode bs i+K , the beret pond: pe should have 
7 ae be ot the frp position a: en geros rainy wile esi 

peoth + diverges Your the all-sero ani ear 
ie unmerged ie K hi Z _browches, must os diss 


th Lu 
a . form ar ha vee Ee 
where Uzgel and U, ...U., is aw ine 
Ca y WO Win $ ts itt lee i ae 


oy the wll reve ae: ok wode i vad or hen Leo © 
we have Own oww Ah oe oe S of One re 
with lenath K whidr t TOO: (ye 
eae y KS Oe). ee j Se ee ois of the 
war bes ee ee bast 8 dS < K-2 and we cowwet 
have vues of KA co oe OS . Theve ore 2° 
suOW LOTS a wS all)=- ah thas Case. for the 
case Vy K cousider the binar sequorias of the form 
UL : Be 8, oo 
ft +4 oy (€-4) -4 Me = ried | 
fue © arian the wumjoy Of sun Diary ULULE Us 
oo Hee .K-A4 we have Sas“ of 
Legs ver frome al{-ze1-0 h at 
ia ee oe the sf hhure, at nade pee La be 
Then K-J 


ah) = z= cL -5 ) 


b) Us: the above results a have 
i, (L) = ps alee 1K, Ett wel F SI 


PAV ATS) hei: 
as £2! 
2" 1" Tuy! #2 Zn 


y a See pst AK 


SSS 


or T.{L)= L*,i* 21 -GLy* a 5 TG V saa 
2. Ao SI - Ki 


KAS ie K4-f 
- Bae BES hace lie ava L ‘TTylt) - ae 


Dewy e 
K ial hae Views : 22 
i, bore WZ fe z “ow 
pee gS b=4 
Trberchamasng summat, tows we have 
-4 p-2kL-4 fie 
ee) Pe 
; evs ee 4-f{ =0u-1 
c va KA i Soe saoan ee kets 
T(t Le] 2 ut ee) FF Fe 
I-L | cei ish Q=4 124 
: “u-L xh 
T+: E(t) ee OZ, Li 
eas q-2L S-L Oz 


eik[s-L-23 LK L- =i). LO Sony OF 4 
[ 4-2 2(§-L) (4 b=! 


eevee ie ee gee Pe 2z'P 
1-2L de-k 2G) 4-ZL 


- if 4-314 212 K qe 3L42L if Ve 
(4-2L)(4-L) {-3L4 21? 
and fiwally 
Tr (L) = L*( 4-L) 
4-2L4 [Kk 
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(QChary ¥5 st of gh ¢ length L-{ coulaius 

Sruary Tssprlgs ts which do not 
ieee 1 lcdhee O's TS oars a. att) for £70. 
for let ee ee Bee abet! Lt Lok 
ond aired oe reins of the ole 


aoe {te-s pa —>) 
iy a least K~-L nce Oo"s ae the frei 
first K-41 
uy | Etsie git f cn (g phe fe sich vd 


K-J Guid BH O"s fs © dt et C1 ER Then 
atte 2tg2%K_ 2b-1¢y g2-& 7) 


aud therefore D6 oi 7 mio : 
4AAS Since K=3, R= 1h the form of the, encoder ss 


S ee 


7 je dato digits 


or 44 2 either OordA 

i 4 

0 for}i = 2 OA 2 
A= 4,2 


Fa ey inti cally all rye, ore o. Te we mort a i? 


mw to 
porticl » ri ara our out ; if i = ty the ie 
Bs jou =e te Ths from stl 64 our 


ust be O4 . This Ree Kes 20 rrr gz ie 
We aa bog 44 to tsals oud shail be wes this t bes 
qut ai and 422 =O sh Scrum will wil be 
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4 


Therese ve 


6 perth « DEL 
pete Ge 
ous eres 
fo it oF [O7tTl 
={DL 0 DL} Et o =A&+b 
prt oO Ut BS, 
fo or. Ou prea 
“Or 0 OL ) be oO 
LE Oo vitt [oO 


From (1) & « (I-A) kb tus 


- 


L 


Di. 1 


oO 
.@) 


a 
i ie fn 
MLE -DL 4 -DL O 
[-DLI oO 4-DLI | O 
det (I-A) = s-DLE + LI (p*22-pi-p*i?1)~ 1-DLt-pez 
= 4-Dii(14L) 
a fy-pLd.sLt-pi?z? Dl? 
% ee DL doo Pe 
4 -DLICi+L) [ DLI Dete fole 
and finell PDL 
t } 5 = Or1 DL 
4-DLTCi+L)| Dit | 


TOLI)= DL % = Dz 


D42T 
4-DLT(44L) 
ue al bal & ) which js exactly (4. ro 
{-DLT(14L) 
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418 From, problem rAN | 
lo 


LE oO] 9 27.20 
A=/ DL O DT |=L145 2 DD: )= 24° 


Ace Re 2 Ub now find, the powialuee of A’: 
- oO 
det [a'-at] =O 2° ae ca. DD [re 


DI Oo f-) 


Then, 


Dl lO a AO, rye DIVA ond 
1 D1-/d214, 407° 2 
< . : 
We cam write Alas 
* [70 OF 
N = AAW = ial OomolN 
Lo 0% 
where H is the wotrix at A 
/ g Z 
Nas ace ial M we ole 


Iw eee sh : p Ra or 3 should have terme with lowest 


rv ortional to pve and the element AX 
ae eA Jeast ox fast ox De | a 


1460 


XN 
cs 


\l At Va eS Cv. So —» 


received sequence : 40 10 10- O4 Old 3O 


Cin davkew limes: path for the correch Sequence, transmitted ) 
| \ 


{ 
00 — Le 1 « { 
ae ge, Se ma a be 
Ol s > Ps 1 t a ' x j Y 
5 thas de yee y 
\ 0 /x Nz any oe 
10 »6 oe 2 0 “\0 \q x 
= P) 2 , 
i : : \ \ \ f \ \ i 
° 2 2 roe eos 
Weights of bvawhes with vecuved se Mole 
rere) je) 4 zZ y 3 4 3 - 
Ol e 3 ° 
10 e 4 e 
lle 3 - 


Trellis with survivors! Hamming dictamces ( lower path chosen 


in Case of tie). 


4.20 


(a) Suppose a\\-2ero tapas . Suppose MAK J = AT/2 

Frsm the fact that Pel) 2 Pol-5) we kno chen 

inpud ts “0” there jis more possibility that the 
14] 


oukpuk Ts emceded ao thet xt posi tie meric . 


A i/ 
Divas = C= pat) 23 \° 


=A S/ 
Aaa E ve al 
2 Ag: Pol 5y) PGS) = Toyo) - zh 
R=-d-A5/3 


shana d+ co ey ree Ag=ad ' dee MUN NEAT integer 
jr t jatar -- thy = 
Gy = the number sf such event 


A = Poirwise error probabili y ter um sncorrect poth 
at Hamming distanca d  frem the correct path 
Uupen remergens , 

= ein) Sum oot all possible product's ( d times) 
of Pe(5) whos. sum- metrics are Megatine 
plus half of all possible products Cd timed 


of Pols’) whese SUm-metrics are 2err, 


= vo oy o 
a ‘: Ap Ply) Pal) Pol ja) 4 


Wo 
2 / 
+5 ds Pee 
= {Hat ; 
A260) = or 2. Dida) De (4.4.4) 
= ie > F >i S 3 atd.39)D4 (+.4, \o) 
+ d=d¢\ «+l ye 
= = 2 bid) dD 
PrEjm} < ra (ZS zaldad) Pa (4.4.2) 
2 Li : 
dadt 
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cy yee eo 
Thte@) = Pp(-\) 27 + PO 2 


£7) = —Py(-A\) 2 + PoC) 


ax = Po(-\) 4 | a 
ree —\ 3 a 2 O 


Te) ave) 2° ter O¢2 4). 


1% = min . aN PR pa 
Ze ee) pa) [BD + p.c0| iy 


= 2/RWRCD = alpa-p = 
vhing pts cressaver probability - 


Here Dp = Pi -\) < 4 


<)  Preesf st the first pars Scr JT24 ts omi'fad, 
a” ae | 


d 
ee ‘|° = Z, G) (rca) ert mEAO 2\" 
= =) pa)? pc) ae 

d d 


a2 > | 24 PG) are Se oe ee 
Whan dts add; 
eo Ss: io 
oy oan Paci) >. Zz rial (4) Ee esc)" | 
nit! (2 = ( aie Pc/pacv] 2 J a 
es a at Buy- ce Rais) <\ ) 


Whom d ts even 5 


" 


/43 


vee a 8 i 
a ( i) Po Fo + yaa ( =) P, (A) Pt)” 


Tay F_ ze 
ae . : re Fi 
tz ao) Poy pcos 


4 oe 
aie eA oni a z a 
= (ac = (2) | BI 


Poe bid Py 


N : N 
Be dun ny = CO a ba 2 VP Oya| Xn) POY nL Xa) 


m=\ 


~ JP (YnlXn) P lyn Xn) 


= {T u Pcyalan) TT Py nl Xm) 


)! 


\) 
} 
Fore 
M 


— 


2 Pere Palyta’ 


i kK) = = Pro (Ja) Pal ya’) (2.3.18) 


——- 
— 


xp {| —(— bn z4 Palla) Pply la’) )| S 


PeG) < 2 Pr} error caused by On ong of ald) Mmesrrect 
park re Bhattacharyya distanu d | 


ald) Py 
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To define generating functiom > 
D The Bhattacharyya dtstonce baturean amay ture 
ditterent inputs ie GO shsuld len sang too 


al x Ee. 
@ Fr me input binony dala , me stonuh shad be 


transmitted +o chanel, 
Let b= # o& different simbols from correct poth. 


then re = a Py py \* = ie 

whore Ve 
amd Cay 2 Aik) Ph < =. Ath) Dp 
Sinw each branch ha oqunal distomce , 


N 


S I PUIX) POLK) 
$e 


Gow, iG] 0, 
, . : 
(4.413) PL) < + ees ‘ 
yb : 
Li 
ape © 
Le 
Co ee Te 
LT 
Ty ee +L U-LI +1 en 
7 J-LE -VL LLL LEt+ Lie I~ (LAL OL 
> 2 + a = 
2 1.1) = L , ECL ral a LE : 
frog. a7 ye a Goad CED a [\-CL+L)T] 


14S 


So) = = TtL.y) 


<=; Ls Pe 
where Co 


are 


D Hard de GTSTO 5 


Do= = STIIES Ply|x’) = 


= pipayy + Sp 


~)|on 


>) Soft decision ; 


_ 
—< 


3 


givem Ro Fallows z 


L I— Cd.+p,2)]? 


= > [arp £ 6 2 


ee 


7 


shew cack: 77 ha, opral mieae')) ay 


fron C213,N1), 


Pos |” GT FYB dy 


—0o 


1) 


}-2L+L* 
2 ie lel) Ws ee 
1-2L 2 (—2L 


1 


Fb 


PY Wa WW /4gre | = 2p 4-F fans’ 


a C fee MP f-sar (IWATE dy 


4 


» 


P 
5 


4.24 (0) » 60,4) = %(D,¢) 
3 (D,t44) =D 0") + D%4(0,0) 
%, C0, 13) =D , 4 D5, CDt) 


lo 4 
(of) <]D o DZD =z A 5@H 
scm ff 2 S[yoeas 


(> 
% (D4) 210 : 
fo) 


by (Oba) = AGODA) = A E(DE) = A BeOD 
F you) - Za’ gto) (E-AY E004) 
=0 tzo 


fies oe 


-3 1D. iv 2 
(I-A) = ee et DD iD). Pp 
ee fe) -D eos, D Dis ES), 
25 = =4 
TD Oe 7 (0) :[o Po] ZED tH)=fo Dio f(t 50H) 
Lez t=; 
-D 4D D : 
2 [o D* o | Do ep 
(> -D td] b20 1-27 


(c) [0 9*0 | y(t) corvespowds to all peths thea yekn to 
the all- ro, stcke at t+41 ‘branches afte diver img . Hence 
fo of Sx (0b represen all paths thed rebum to the all.sro 
a oe + he erm a decision is to be 
ha: ow vic T branches then s 
not et remey : pd ba sis so ckeods basal emo 


paths Wave transfer function 462.2), 4. ) )and & (0%). 
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Howce C4 
eG) ¢[o D? oSZ, LD) +[ f aa %(D,%) 
ef 


(d) rom 4.4®6) we have A? = DAZ aDA 
A* = DAP+DA*~ DPA +DAC,D°A 
_ (DUD) 24 DEA 
A? . (poor OA]; DA 
= (D4 2D*) A’ a(D5s 2) A 
A° =(D*s 2p?) (9A DAT ¥(D% v*) A’ 
For val: =(D*+394D2) Aes (O44 20°) A 
Z ZOO) = FA ECA) «(2H AtAte Ara AS) ECD 
te t=o 
-[14A+A°sA+ DA (D440) As D7A 4(D4s2i5) AZ 
+(0740°) AT CO, 1) 
-|1+ (440+ 2p2+D3) A+U +202 304 D°)a*fEO,) 


[oO a6 : ip oD D- 
A-|D o D[, Nh 2lp* p v* » BCD.) = O 
De. OD [D? > D* O 
i 7 [ot 
A%C9,1) = Dp? 5 A 9 00,4) = p‘ 
3 4 
Bs | v J 


G 
Honce fo oO o | ie $(Dt) =fo p*o | z Ag COA) 


= 8%; "4074408 > 
Next vote thet (D1)= Af "eto 1) 
D2 
| = (20%; 19 os] ] 
D 
[43 1% (07) - D* 3D°sgp7;p% and finally 
(4.7) € 2D54 1SD*, 4375 Spty D3 
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425 
@) wWrthad loss generality , let correct path ke all-2er0 pot. 


Pets. L) = Pri the decoded pecth lemres the correct path — 
oF mode 5} 
= Prj the paths that initially leawe the wrreck pot th ot 
node 3 ad geo to seme stute K in Lb branches } 


: PAU the pathsSc te sta x tn L bromchedl 


Fe Pr tho paths aA ge to stich agar ea Lramdws) 


al\ X 


pra | gamerat ing, function wrth D= Japarp rates y 
"he te stake fa‘ ic bronches } 


2 x 


= 2. ( = f[ \\-:-1 
Ze 3D) | eres 7 ENE CO a a 
b 2 
(b) cD ,4+\)= $,(0,%) +D § (0,4) 
3, (0,44) = dD’ §,(0.%) + 5, (0) 
35. CDN) = D3 cod tO) 
La Cb +t) = DS.(pD * bd §y¢0.) 


30D, 44) = A Fat) 


ohere Ao ] 

a Daas ee, 
oD... 9-0 
bon De pp) 


2 3.0D,6) = Citys) §(d.6) = LWIA § 60,9) 
Lvvviy A® 30D.) 
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Using the Cayley - Hams | ton theorem *, 


eee let. 
—p* Ss —| 6 
° —D SU 
=p eo s-D 
7S = 8) DG bh 6 
“> 5 2 = Ss =p 
-D o s-D “Ve Go 5.0 


= 6-) (s?_ps'—-Ds) + DT (-sD +p?-b™) 


Se i) = = Cb) 


A*= C +b) & ne Ey DA 


= 3,(0.6)= CUI A® R00, 


AP = Cid Ar + (DMD A® + CIFDI00S—D) AL 
Pree Gees Depa | A2=-f \+D> Dept pep Dt p*] 
De D p* D »* + D> p> ra ae D D+ o- 
57 D 2 D De D4D* ae Db: D+ b> b+ D? 
Pape Dre Da L ppt Dap DD Dap 


= (ued) CUNT A® 300.4) + COED) CLUNITA’ 8 00, 


+ (1+D)(DP—D) A § 0D.) 


= (0) (2D-42D +0 4D bo + C0) ire ep) D- 


+ C1409(D°—b) (2D) h* 


= Sip +90 26D +6074 5) 1 
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4} 


STATE DIAGRAM 


Hovifieo State 


%, = DIt,+ 9% 
b 7 TY +r Pah gg [ 
i. out + DL Fe ory, + Dr = amore 
: ' 
_ pt EG HIG, PFs Cz) 


_w 
T(D.L)=D Gyr 0° 4.4 07% (4) 


(O.D-( =>) De, - %, <(D51-01)%, 2% -LOT-Dmyy, 
((2xL-G) =7) L%,.%20 => 522% 


Solvin r We os Det 
4 fe Ye ¥ % 4-291 -D*12 
ie aie Dba %,. pei? 


§-2pI - pe Lz j-20r - DSI? 
and 
TCO,3) < S [pr (D°r-oT+4) 4051 + Deze | 
{-2D1 - D*12 
fall 
TE) On Clore) 
1-2pI -p°t? 


From (4.4.13) we have 
(fat Ge also 


b Or |x=4 
D=Z 
00,1) . _29°(1+D°I) 
at (4-201 - D§12)2 
ely ee ee lee Zoe 
2 ($208 - D#re)" bres Cl 22-252 
< 


4.27 we (wid blb-kr)dt 
50 
Elwyn] = Eff mernprbla-tr)h(e-gt )deeds | 
= ae: [nce nia) | h(a-k1) hip-4T) doe dg 
ai f “he §G-p) hl-kT) Mp-j1) dade 
z be [hat-kr) hla-sT) ds = hb)A 


a a? | 


-_ 


1524 


Choose ¥ Syeo pet. -kT)dt 
(* x(t) pt-kT) dt 4 f nit) p(t-kT) dt 


-& 
00 


Let My = ( wi) pa-kT) dt 


Then : 
g[%,%,| = de a0 perkn) pt-gT)dt = we S, 


= 


leva Be apc - OF Ty? Pome? Met ao 


N-J 
x(+) = ye ik 


k=-N 
an fi Y plk-kt) dt = vs fh t-krdt 
x¢ T h(t-iT) pCt-kT. 
re Ske-jop 
ia Zu he 
ke ~ 
s 7 Uk; 
pok-wes 
L-4 ee 
= Z, Uy; hi 
iso 


Simte We GASssuwme twat Wt) = a for + <O 
W(t) = fr tz 7 
This gives A he, “intagrae ieee f view out 


% - Z hi Up-i + % 
4=0 


We have the conditioned Probability 


vs 4 ~-%)/hb 
: pole ie a 
Ce He : J, Cia) 


Tor tuo Sequence y and w with error = F£(4-4) ug 
have the paivrwnse ervow se lity 


(8) = Q(z-zyfau ) 


ks -N 


The Wax VL likelihood dete coy chooses Ku tet Warr. 
miaes Pau ¥1¥) when y is the observable . Hence 

A -4 a, | z 

we mex 7 Clu) win NY-Z Cui 


fad ad 


| . 2 
= oleae” L [2Cy, 2(y))- tzcusl?] 


wo 


bet 1 2 tf 22'S 15 r 
= = (,2(4))-L0 Zu) "22 Z y 2ty)- cu 
mF 4) b / The eae 
n-1 L-1 w L-Ln z 
a2 7, hy Uy - (2 hu t | 
No kz-v I $30 oe M, fro f ) 
v-4 
7 24 Xy Cy, ; Up  Upiyy:> “4 ¢¢4)) 
{s) k= N 


/S4 


This welvic shows us that the maximum Iikelihood demoduledor 
com be redliged with the Viterbi alaovithw Mext we have the 
vit evrov bound % G8) ye . 

{ < re hi © ei) 
Vy é < z w(€) Tt ye) e ° 

fo £22 we hwe Tad +27 stodes with brandy velues def neo 

by ~4 (h,)° oe S fothy 


A= & fe o, 
ACoA)” 4 ak) 


y 


Cz" 


The error stede diag cate iS 


CG biz 


This veduces ty hee) Ih 
ro) at/e ad O 
yielding trenster fumetion +2 a 
Wo €.4<0)- ost 
J the bound {~(byc)I/z 
< Ww ~ ~ 

. ae he + he) 
Re eo ahd S e “% 


= taf d-brO kl fis tout? BG ahha 4 ahh) 


1SssS” 


a 426 Assume Y= fu, +f Uy, + where Elyit "J Mebey. 
ey: f J Nef Yet 
= foLfrrgiteet eles [unas ft hi 
: (ao Ut fof, CUyit ea) +4 My +f Aya, 
= hols Wut + fay Hf My, 
EM tes fen Hy of Hd) | 
= te] tS b+ fof (Saas? ‘ui) * fr L; | 
Z we | ho he +h (Ai, Seid) | = Me M44 
Wee Y, = hoy + nly, + Uke) +My, where Elm). sath 
(b) and) follow as im eg 4.17 with ho: = fo and hy = =f;. 


Then ~A.(C244, 
ue 6.2 No 5 ; 
od, (6*462)/ 2. ¢ er es 
[yeh Mc Ph SAF] 
She 
ek aZ@ ° IE 
of -2 2. hey z 
[ig TR RY] 
b-4 
GA) Assume Y = 2) fu. +h, . Then 
<1 *=0 f-4 #al 3 
1? 2 ‘fi Yeu = ae ap f, Aba; rf, | 


Now GS suming } hb. =0 ey lil > ZB 
f me 9 Y »<O cud te Go 
We have 


H(0)= Zh: DY = £(0) £00") 
(zp v)/zeo*). yk 
(ZAP) ERP”) =2 244,28 


ond this is cousisteve : 
@ Dawe OS Ww provlen. 4.21 wily s = fi for a0. Fri 


/$s7 


( 
M 
| ite 
Co 4) 
20 
> 
+ 
w 
Ne 
AS 
on 
_ 
> 
x 
ae 


are SE 
(@) For a nm-coherent MFSK , spur ed matched filter - 


envelope detector oud pus for each cht b ( or 24ui walondy 
sums of the Squares cf the i phase and yaad roture 
correlator out, p uts ) are Ss ummed to -ferm the dectst mM 


Statistics . 


let 2 = a°+3;° 


output amd Ss) the quadruture esrreladin outpul Sv 


ohne AS is the Tn—~phasa usrrelotor 


the zh chip. 


Using Chernhoff bound; Prfe>o} < Elef*\ , pro 


Csea preb. 2.10. b) 
oe = = (2! 
Lara Pr? ey 2; am 233 ea Hey = et 5 ea 


2 ' 
ae L exp {pa (2;'-25){ | = ia E [ exp (p(a5’-a)) | 
3 ie 
= i) EL eap{eta3*e yt) EL expj-e (as ty9s 4 J 
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amd E (09% E (yt) = 2€b/ Ned 
EL expf eum sast) = exp(- map HE) 0%) 


EL ep § e( 43 +937) | = V\-2¢ ; ore 
Letting Sf o<A<\_. 
P5 < TT pe mp = a <2 
2 ag | . 
< men Lop eeplo re ell = 27 
ole z= men cage po aiy He : 


(L) Since MO more tam > paths MRR of ter K+} 


bromchee = ad 6 unger) ey om COS Ns 


than jel bit errors) the bit error probability ; 
aneroged over the ensemble at K code , ts bounded by 


va : ye oe K 
= Ps is oe 2 >) = 
ae < a 54 1) 2: Pas < = ‘Shiv Zz (\-22)° 
4) 


4, 30. 
(a) Suppose +he a\\- 2zerc signal GOES Tnre the encoder. 
Suppose the decoded path 


Tn the Trellis dtagram , 
diverges a tha cared. path amd vemerges af ter 
JT branches. Since the distonces befurean ams 


toro differenk signals x,%’ ore same for «\) 

i eo; the labels on the diagram wre either 
D ( when erther me of module-2 adder outpul ree) 
or | ( when both of adder oitputs ee Go) 
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Then rts state diag rom labeled wrth distanws from 


al\\ zero path cs shown kalo: 


Using Masom’s rule ( see SI. Masyn | Feed back Theary 
— Some Properties of Srgnah Flow Graph s ; 4 Prac. Re 
vol, 41, mo 4, pp. UI44-1159, Soph. 1953), we lowe 


| a 
= — 2 54 
DVS 
< N sili 8, (= redo 
sith pA=|- PBS pe? Cy oe 2 Po ee 
me x eI is rel 


oho LL: # of all possi bls forward paths . N= # of feedback lsops 
mys £ of possible corbinotims oF h nentouchins lwops _ 
H = largast possible # SS mom touching loops . 
T; = Dystance of jth feruard path . 
p.th) 2 prs duct of dtstanns ot jth cambi nat? on of 
h nonteuching lcop> 
A> = value of A for tw part of signal flew 
dtagrom not touching sth farvard path. 
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Example of Masm’s rule , For Figurr 4.10. 
DLL DLE DDE £ Pia Dp cl- a) 
|— PLT - DL- ER SPLIT DL-LL t DLT -CDELY) 


TCD 1) = 


Sete Peep) 2 DeLee CoS 
I pLOab Sos 


Therefere ue have te our sti diagram , 
1D) = i p* A, + DF. A, + De Aa tDo Ag+ +” d.| 
where Ae has the frm; 
As = 2 Cg D’ , Cg TS MEnnegatiwe integer 
Since all \cop hog the distomce D™ uheanng mis « posr tw 


ibibo. gid For the same reasm, A hag Ha Same 
form an Qe 


[+ D- bd 
eK Teo) | a eee) 
D=< (+ 2-b(2) ty 


-(b) Fer the gemeral class &f semj-orthogonal umyolutiona| 
pecader” Guth hk = 3 ic 

distance path mm the std diagram hao distin D0” 
whan only oe | ek deceden . Therefer 10D) has 
the follmoing form: = TCD) = DEF 1+ DacD}7/F 1+P Oh 


z* }+ 2 ald) 
ee TOM yaa > 2 |+ 2. b(2) e 


J6Gl 


(- fo-fiy-fo- Fr) 


0 


otfite-f.) 


(b) u:9 ields the outpuk sequence Zz where, zy = f, +f, for 
alk. For I , where a 7a the path corresponding to y! 
Aiverges From for four bramlus os S helow : 


Eh) hirer) hoop) lh aD 
Cette f-40 (for fuforfs) oat tether fs 
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(00d act 


abl ec 


2.38 G5 6-7 

Iga Gel + ct te Vier. Ye, octood| 
Tr= beth + abe 71be 0 0 9 abo O 
Ts 2 abl T4020. 3)a01 ©O 0.6 ay 0 ©) 
Ta cocte + tT; peg i Ga oe c* 0 
Ve 21T, 4 bite ipod es 20 
Me oars, as | pee 
: TLO O bIO O O abl 


T1s bel Tye abl T / ; 
ae AL +B & B® .lacloo0 00 o| 
T(t. A) 8 
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Note thet Liz “ZW s fF alt ah YoGeh Pe p74 fe a(G4f,) 2 C2 207 
a 26° 2f\ + 3 (fork) 
PC U + u') = ( Wz-zN 


ae ) < exp {- NZ-Z07/4ab § 
: . Ite +2f "+ 3(forf.)* 


a No 
a Sot SFP + OF oe, 
am & 
Sho +6 ln 
abe No 


where we used ie = i naa oe Iv = fof. 


C) For aie | ee bacaldaaas w wah r the forse error bound AS 


_ Hz-zN2 
Gury!) = O(iz'-zi/ffae) < came 
Zi, ~ Zk) 
= TT De 4Wo 


k 
~(Ze- -§,) ~ ie Ay 
Thus We Sore Ve branch vcdues z@ © @ amt fi) 


4, ech of the branches weth outputs ZL, cae io ( Here 
we hve Ke Lhe! = fof) Also we use the paramrcler 
Lt fer tose brouches Corresponding th "I “dete, Synibds. 


) Me tel. _¢2 :: aay 
ais fe/ ee ree £.)/Ms 


ze CE 
S) 


Then we have the modifiad s ode. diagrem ow the 
followina, poae 


/64 


We've owl interested in Tla,o,c,L)= bt so 
T (a.b,c,2) = Lo Gobgoolt - eG. -A)B 
= dbo T yy 


Where Mig is the fivst column , fourth vow Lompouenit 
A Ca We 


4.32(0) Lek Si. = %, ve the stede Gude hove Hie 
dvo\\is diagram 


XK KK SK SS ae 
Y=: eG b c. b < w (oe & 

We want 4, ima the pat with oukputs z Peck monivuigns 
Pt ylz) or log POp\z) = = log PH. ) 


Or mimic M(y,z) = = wh. Zk) whore 


vA(y,2) = ‘[()] / 4(4) pis given by 


OG 4 ©. 4d 

61-54. 2 j 

{m(y,2) 5 > bl f oe 2 
Cc 2 4 O 4 

S14 2 3 O 


The maximum likdilaod dole sequence is then 
%,=0 6 X20 X34 ; Ky 20 : %% 2 0 for k>5 
avd branch mekvics ave shown im the trellis dieqvam 


16S 


©) Ye (x->x') ¢ Zy Poyx) pop) 
x 


= TT 25 ip(y[Z) GZ) 
ENP PCY 
where Zz bX, Xe ond Zi = E CX Xe) 


Buk X¥2=O TO Zz for cl k- Hence 


—. { 2 (oe 
2 Vplapyre = 20pq Z’=b ov z/ad 
7 49 y Ae 
5) 22a 
= D' Zzb or Ze 
D* Zee 


Where De | P4 . The stoke diagram with Dandi is 
I 


vt 2. , 
yal Avancfer function TC(D,1) = D?I/G-DI) and 


@ the bound 
: % ¢ 9T(OL) /al Fe = cae “vy” 


(<) This DMc is the Sawe as two uses of a BSC with 
cwossover PD, Hence E.<¢) Ro, & (0) aud C ove all tuo times 
the Covvertpouding Values for the BSC. 
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4.33 @) Using Lhe <came rectonina as iw problem, 4.27 we 
Nave 


: Yk = Zu, Wy Mya Seay 

Ussumed ae Zu 4% 2 S.+h, 

where ‘ = (mein ptt- LT) At 
Wht - S eat) p-kT) st 

mM U @ vclude, S is, ye Sian (eu for 
aes d yc as alt The eee | gpl 

7 eee iu the ceeision, rule we com 
ne Sinp eek to be sent cz, y= Sol wes Ce ere Crvoyr 
probally Plu wu’) vs , for the Aw ; 


Hluoy')= PP nS-yll i3-y¥0| w is sent J 
> ZPlylu)= 2 PyIs) 
K 7 


where fh = ty: PCs) v PCIE) f 
-Wy-£ 117 
78) =) 


~ Pepa (ey 
Vet ine an vadinte, fuwcion FCy) oS 


fH =f fA 


- Il y-SII/o 


/67 


Then f 
Ry) = = Cy) Pers) 


FC) com be bounded as 
it < ous") yeh 


t(¥)= cel s). 
O« oak fA 
Vence Blyusu'l)< = pcyls!) pcyls) 
Sow a, FCuRY 
(y\s) = a ee “Gi -8:) No 
ite hz-W mies : 
awd Bails). it @ “ae le a BFF 
POISE) we 
So tad 


B(u-y) € T & 
} j=-N 
Rs 2(3/* Si 42s, x )-C Sf 8; +231)" 
= Gi -&) (8 + 38; - 4%) 


LEM (obs meg) Pf Bhs (Wg Buca) the teal 
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Given u and u’ there ave Ww (4, Ki!) places where there 


m be a bit error. Tp we som over all & ¥ 3’ (fer 
infimite Sequences ) we can Sound the. bit error P ily 
a coo =—-Kk/Ghb 


where - so (-ke + Dhyly + Thol,- Lhohy ) 
‘ ; gk hts Zi,k, eahh, ahh, ) 
5 cer techs Lhyhy - 2hoh, +2}, ) 
. AG er - Zhoha + 2hoh, } 


= gy both (he 2h haze 
veducad slike diaqvawy 


5/2 
1 +*# are 9) bed ape 


i~ 45 =. 


> 


104 


Stike eRUciione : 
Se arcr+{, s 


s ALC 


: ( /2 
ales b,c.4,¢) = (btd)s 
pa (arc) Cb+d) 


1~f/2 


// 


17O 


Cho-pten 5 
P-L) In problem 3.2, channel ¢) we laa 
Folp)= pln 32 and E(R)= ln 32 -2= C-R ,0¢REC. 


p 
~- 4 lu 5 
4 ci. a => E.ce)= Ali): 3h ;0¢REC=R. 


3, SIR) NE (@ 
E. (2)= Polp)=0 ln ¥ We eae 
P o<e< 00 Nea 
R = FeCe) = lw S/2 
: > 
Coluty & 


For chowndltit) we had folp)zpl42 and so 
reer, ts the Same as oe with [x 3/2 replaced 
Te ae 


(b)'In prodiew 3.3(0) we had £,(p) = ln Q se agam 
werything is AS w gat ©) wth in replac WZ, 
Iw Bal Wr : 4 


Ek 
Fotp) = pimO~ 4p) baf(@-s) pIF 45 AP 
Eo(sje wQ-2 tw)(2-1) [p+ iF J 


CelnQtlQ-iplupr pup bo Fes h®) 
fa) 


ECRe hl , ReRK< C=fe)/ o 
Rez Fol = 

Roz 4. p/P ,0<k<R, r F(e) 

— eT CueR 


For 3.3(<) use the vesult fer 3-3) with Qe2 and 
Une yesult fow 3.3(b) with Q= 4. 

For Fe(Q) we use jut the fom impute Ay Og, Org Oe 
ond set 21g = 0%. 


47/1 


88 
64 


E(2) — 
for(a) 


ECR) por) 
fer Feo) we take the 
vata envelope, <u ill eve 


(c) frow lew 3.5 we ha 
ve 


= Ww AP e e = 5 


lW(1s8) ay & 
(7 


(d) frow problem 3.6 
+ We 
ave, Eo(p) = ba 2 lu (4 B 
I72 . : 


E(4)2 wd - ~ ln (44 Q") : C =(ln Q)/z 


F.Ca)= egaree sOLELR, 
saa lw 2 lw 44Q_ 
= R= w2/p {lwUrat my | 
z ) 
G: F<p)/e 


(Vee for Zeese 


C- for Ga € Re 


ae OL R<C/y 
GY-2 4-2%/ 
ty Fl), C/z : j ss ,%< R<c/r 


Re” £ce) } We -fe)* 2t4- tec’) 
eee 2 8 e  =—6 cece 


(e-Ve (safely? 


83 (0) W22" , Reb. Wort Lbhe. 2 2 
Mb (Lik-d)n (Lek-sm LtK-d 
(b) 2 


“KES () 
EclLQBc i _@b-s) DEEP) 
: 7 peppy al 


wat bed Bh sn ey KELME SAME OH) 
(74 = 


ie) ~ MWKE, ©) 
Te <_b (26-4) @ LtK-4 


z Le PR-F I hes el ea . 
© het /< = L(2*-1) ud G2 78 
) ie 2 Pl fpya-p] . Lik-4 >% Or) 


So Wess 2 1-6 an therefore, (“L < K 
L+k-3 L+k-{ Lik~4 


Wyck exp | -M, (1-6) £0) | whore 
ECR) = Fo (e) for Bleed 
Rofy = 2 = folpilt-6 
‘e 
Q) Miniwising with respect to © above we get 
% < ke seal fe N, ay ee F(e)] : (@R: Rp) 
©) from ©) = £olp) G-) => Le : © £6) 
{~ 
or ()-0) £0») = (1-e) ech = &(p)- & Pr 
wheve a7 ¥,(R,) over © is the oe as Mc Xin) ging 
MY P18 EylRe)= Seer) Fole) - Por /o-e) ] - 
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4 Let Le keait » Keke ; O<P 4 
at - Efe)-p tote) y 


EC = M4) Folp) ~ K (p) _ KAO -pki Ep) 
*e) Kewits K-4 ery K ECP) 


. Fp) -pka(p) 


55 (a) Note Yat tre terminated code is a special cose Of 
lock codes . Thus : | ee 
ez 


% 7 apf rr[hap @poory]E eres ey 


Lt K~{ 
(o) 27 axp {- n(LeK-1) | EspCPe) +0(K) | t 
= Wh LK [ERO] ste] 


= 1 


where Fso( Re) 2 hip)-p &¢e) Cea. 
Be = fylp)= OK ofO< { 
ces Lt, tA 2 = 
K K-4 1-© 
(c) te S 1 ~ Pe LEap(n)+ 06x) Ihe ics ete fov all e 
= \ = \ eS f 
aad hte Fag) = BR gl HOMO Or 


The minimum occurs at 
@x(1- EW \( 96) -[ Fol) 1(-EE#) 
s 


17 


=> ole) - 2) « FoCp) 
4. Es Cp)/2 


ee 6 (6) devee i the minimum distomce behveen Closest 
va, for'hne dormnimedod RT ana at treoked as ee 
Vout, Code, We hee 


duuin block) = drag, Ceonvolrctional 


(b) deren = __ doin ~ dun ¢ DR) 


(K-u (K-41) (L4K-1) 0 
aes “C-OM, VU, (1-0) 


where "v= =O. Mewimigi m4 with respect ” L and ® 


Sere gwmw DER) 
(K-4)~ oes M, (4-€) 


© usi be Yobkin Yound a 
oul Oe f(s fs) ia 


eve g min 1 Set) ie ele? 
(K-4)m OOecl 2 {-© 2 


ot 9.1 (O) From $20. 5.4 we heave fo > min hep 2 a TG) 
“Bal TH.G) con be lower bownded by ud, avercn 
Prcleloitity oF i abe Orrove for one jmcorvect pot 
omc for $ ila \y whidh vs fe K branches. 

i Lh ah will have, 
Zh Tone ar fara} 3 5(«%)F] 
Noting tek here a6 win, for the, terwinckd code. 
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By assumption Shee £ $(2) and so Hr Z a) 
(bo) 1% & §@) é ——- , then 


ln( “7 -4) 
ped Z _RK- 
? SS _ at la (207) 
b 


bik Rluz = EaeetR) ae R= 2 In 2 , therefore 


Yes ez ; 

- R 

(c) {row Yreorewm 5.3.4 we heave S42 Hp Bean ) 
cwd our lowew bound mlb) contradicts this result. ue 


must Men have, at least me Br ib teknca| code Sch Wan 
aa 2 K 
In(2e *.4) 


(d) See thy Cowolioued code is o specie] COSZ 
of block code, , we have fo input sihaesst channels 
H<LR < L exp [- M, Fonte) | 
where Ny 2(L4k- -1\n aud Kz binr2 


mre L+k-{ ” 
Fe. (2) <7By ln Z 
by(3-4.8) Zelu2-H(d) of Re me-AGE ) 


Where Sv = 3 zi ee therefore we have 


aa 
aie wt <t Z Leben? 
(5 


mi §: ine Ser: arti 
Ba oe 4 Chr ued 
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=e By te the Orgumnest used in sec. §.4 erate the Gallager 


eG) ¢ z Tp) ¢ < cme The) 


but TH) < lab. 4) zai Poe Ep) 
g pblkeadp ,-Cerkye tolp)u2 


TG) : htop Kh) oECp/e 


. BL ER beter] 
ae Ke (44) [Eotp)-p ® i 


-_ 
-_ 


wheye « kk and Re Are ZB 
44Xr 


Since We wipe ix ident: ecah to thot of the block 
3.7.17) , manimizi 


bound _— with y to 
~ WB G49) (e,d) “pu woe 


where (2,2) = Eo(p) - )  o¢ pss 
and R = Eco) PPD, caf 


cud where E(@X)- EC1)-2 = RB < FCs) , 
ho TO) £ er" min +VECe 2) 


v lavage K we heave 
“at E(RA) = &(p)-p lp) li? Ul-p&tp a gf =? 
bat from Re bipdrAVe)/Cray[R we hare 
178 


js Opel : 


= £7lp) 2 aud with this we get 
£,(9) - rp sy ae SVK = 0 


inet 
JHA 

which anes ws 2 PRL YK) 4 

Ey (p)-p fp) eS, 


Re(LN EO. _ BPPBO _ Ky 
rn VK pkG-Y) (4- Yu) 
(P)-P <P) 
finally we apt sui eels 


Re Fl) when =r faa bag or B4)<R<C 
“To hee thet Yr yelwe of y gives A winswum we hae 
Aipecaal _ - pe - pee PRP L- delt- Yee] 


dn 
+ U- Ye -W)B . - p? (4- Ye)” We, 


T G42) RIE 
Dud so ge (44) ECR,2) = = pR(s- thi) = Eo(p) (4 - - if) 
— ~Kkb [E (p+ OCK) 
a4) Zz Ltt J 
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5.3 We want to show thed linn ike levit - KE . oO. 


—p Oo % 
cing The, sae ore as iw the proof o¢ orem, 


keyst- ce .. . 
BA kc kevit-eK} e. ra Thy) 
a. 17 KL bp eo We 
2 max Th 4) 
oskekevit-eK 


We vote thet Le) MI Th (4) as “wen revious h 
im the text fat be a eblean, 6. ef bboy smal] € » kerit 3 eK 


(Ul a 7,0 ee the Given Ya k. 
por Hai ci" = kepit § Je $ Aer 75 of 
ies WP espe. shat found mw 5.8 wil Ul 


LEu 
a ih a = Say ¢ theve exists 
cS “7O ot 


Ela tol +0CkK -)] 
MAX oe Tr ob eo p 
Ofkehet-eK 


Kb LEMratoK 
Then Pike Kerit - eky Cherittd 2 # r 
Fe < 2 ig [apo 16007] 


. vé [E@-% EO)+ a+ otk") 
= = (lerig #4) 


Aud , as p->P* we have 


-Kb 
live Prd ke < kerit- ey < (ker +4) ZF e 
Koo G, 
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ete 
5.10 (2) GCP.T> 4, i44) = pP, a i ee Se 4,4) 
zt 


(0,1, i, ,i44) = ee aed” = (OR, ji) 
glist) 
4 OF 5) *= -)” 4,0. 1,4, 4D EOL i) 
fo pts oA 
. ian Cix\) 
A (its) Te OQ er 
aa | 


a aut 4 assume fat the 
wo) 2 pit te once hes. fey wade err er a 
poor ok Zoot rer some Poa leawinw 


S OW ror. umn 


eS for all ‘: iota cl is then 


> Ciat) 


eli) < Z D Ot; j, ») 


bz 
freely ve Cam ex ress the bound in tb) as 


Pel) < z [0 eset %, (0,1; j, Mes 


Cousider z lo rid ee 


bs O" 


o IZ (OL, 4+) 


qc 


“> foD’ ol AW)AGH)... -ACjss) # 14,4) 


iin 
= 2 [o yr o] AC) ACi-1) « --AQ +4) a 
joie) 
Es 
Over the fime-Varying Code ensearble the {z,” 5 are 
/3/ 


independent random variables for aM k=4,...7 and all. 
Hence j 


ae 


zit) ite Z,p) 
Zoo" o| ¥(2, si) = Z lov O]A---A le Al 


a A= =f ro) 
Gut Dede « Lyset" = L (4420804) 


awe ms five, D>. 2 
As| D oO D 
| DL O DI | 


hich hbase Lo the, ty cows mited Mok wher all 
eae We labeled 4 with dS Sond a\l " "one! chide hae 
ave lebeled also with I, The transfer area ov this 
‘averc.eed " stote, diagram is obotasned ey fot 4.6. iv" as 


ee 
L:D = 44-DD)I 
Nob th F foo] gory) 9.2 50k We | 
bedi uz 
fo SA ZK] <b qe lg 
=[o 3 sear? [= G2] 
44) £713(D1) zed 5% € 7B) | 


(A) this follows im viadly. Ye Some wary. 


IF2 


T,CL,t) 


ae | 
p=5 


S41 For & 3 ur hove the following meitviy fA 


t(oA7) 
{ss fo 
,by) 
£C1, de) Kany {9 f(4,) f 
A = reanyctsd) —fCLAs) 
es (4,Dy) (4,04) 
(4,04 4, “1,A,,) 
Pe tt) toad fod 
! £(0,d) {ons f(0, Be) : 
where  o= (0,0) As= (-4,4) 
Ay = (0 -5) Dez (-4,-4) 
Azz (0,3) A7= (4,9 
Se) Ag= (4,0) 
Az (1-1) 
Tor b, oe h ond A ey mM ure 4.23 we 
qc sy “locng “MA a "9 1 
: Ie " 
7; 4/y ty 
ab dad Vs 
Ye Myc 1 
Mh C af Ya e 
Ya b Mh 4 Mh J 
if, fe 
4/2 4/2 
We have: ~ 
all hs Ak m 1) ond (-4;§) ore the seme. 
oll peths endi im (4,-4) and -4,1) ave the same - 


all paths end} im (0,4) ord (0,-4) ave the same. 
all pets ending’ im C4,0) and C10) ave ic sant 
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Hence A reduces to A below 
r 7 
WZ =O Oo 2 
wv a 0 ‘o) O Ve (j4h) 
Azlo Seb thd 43 0 
oO inc Hy @ Yee O 
ne V2 4/2 Ve O i 


where Ao = (0,0) A, (ti 44) 
A; = (o,+4) Ay ait iO) 
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Ld ae 


v be the row ercgunvector associated unrth X. 


Let 
VA kU. VAD oly 
V max 
x2=———— > 
Vianin \ 
Co 2 
V min 
iN} 
Pic N \ 0 \ Vv 
C ray = Vimin vA . Ta 
N 
te A ee ies 
ra rd Vuin © Vin ? Vimin Xr 
o : 
O° © 
es 
iv = V max Vv 
oe 0 \ ee ee a 
Ce ee Oe ee 
Ih--- | ae: v | = ec ened 
: VA 5 | — Vw “] 5 Vmox = a 
° 6 
© } 
We have 
6 N 
6 
; 3 
where [ Means ) column vecter with me " ” amd 
oO aw. 
: oO” csawlang - 
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5.13, 


WWean tsi (Ait\) = 1 (Rt 


When f=2 , the inegual tty holds > 


CRy+\)¥ (Ratt) < 2( 844) Bo+1) 
> kt dita < aed trakitrkit2 
> OO < 2h ht hih, 

the equality holds ff R= k=O, 


Suppose the Tneguality held fo d= pee 
Then tin Q~ Nee: 


o 


rer om +\) = Le mei +| + =e (R= +\) 


at 


mt 


(m4\) TT Ces tl) = tl) Ant) TT (4s +1) 


AZ| 


m 
= m Panay + bows #1) Th (Bs #0 2 on UV Gt) 


\V 


(Res + tomer + 1) ab x (bs 41) + = (h:+) 
= t= 


IV 


~ 
MRmey + Raney + | + 2 (434\) 
a =\ 
antl 
> Fm FE Do Cbs 4l) = = 4) 
A= FORA 


Sincg Z (A+) 2m wrth apuality whim all $2 =6, 


f 
fone LT Wye eee tt ee 
wrth sigaality uw hon al k= =0 , ke A.age, b. 
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5.54 plat $,)= 1 Pl Yel Xe, Sim) 


Sm = (Xaageuayy + 2 Xn-2) %-1 ) 
(4) Assume X= (%,...,%y) 

x! = (X1,---, xw) 
R(x>x!)= B(x, %'| s8/) 


: Sb wUleso so 


8) 


F, CLs) 
where 
is IES fe lelx st) felelxisi) » oO ; 
Ae Jyh Pu CYIX,3) 


Des ime cM ee funchion fCy) Og 


ey ae 


Then %Cxx'|s,8/)= 2 6Cy plylx,s,) 
x 
£Cy) con be bounded as 


f 4) Z gues for amy y 


Pu CYIX, 8) 


* Elxsisus)< s hl¥ix.s/) pc 
. ¥/x,5 
x Pu (¥1% 80) Iw 2) 
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a7 


30 
Te(y,x'lssi) < EZ PuHxs) BOM 3) 
os) o N ae v on 
as 


Bd 2 (yIXy8) oT Ply) xn.Sm) , henee 


n=1 


elem lsise¢ za 1 PoYon Pn Sm) PCa], 5) 


y wl 


ir U/ PCa] %nsSu) POY! XA, Su). 


"=! y 
b) heb 1 have j wdevit components selechd 
pene "ao) : re 


te Cyvlsusi) = oe 7 (%,x/18,,82)$ 


Zz GO Ge) tf Cx, x//s,,s/) 


%, x! 
aera pa ae des T 7 Cn) FOX) 


xX, Xr XX’ Xv x7 H=1 


, S J PLY xn) pal Xj) 


IN 


©) We have x= kx’), qCx) = qo) q(x’) and 
Sty = 9 (x1, S«)) and lev 


Su = (Su,sw) € 74, , 42, A D,.20£-1) § aA: 
wv a 
Pe(arm'g)< SZ, qlee) Spey spots) 
/88 


then 


Let 
Qj; : C4, \) f Ki = (x, S;) 
if hens 


Where 
A = {ais} 


fC x,s) = (2 / eCy|x,5) pyle | 
Then 


Rawle) = C414 ..4 TA” ids.) 


where LCS) is the K 2GE-1)_ Ae) Column vector 


ged 2* obhmwite OPN A 


Q) Frou (8:8. 5), the icin bit erro ror probelhiy bound 


ie ‘e won ad tn error ow for the bewwrel 


aba 2 b(e+1) 2% P, 
oo a) 


where % 2 PeCx xls), with Va ali+k) such thet 


xX 
(Kk 
% « ats F yuasya? ie sre” ic LC3,) 


by He Samed at aca as iM section 5.3 
L 3" fia 44 Ne Ms) ¢ & ya 
Jee, WV is e.z0el MAXIMUM eigenvalue of A and aro 


is the ie waponerit of the lett eigenvector aswccited 
ak x Teed. by ats smatlest component 
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Hence 


2, é ata pie aa 


LS) 4 


aM otner sf follow iW sine mayne with 
the vesul} Hor 6.9.1 whch opphes fer thes 
pahen fi mite ceca Cake. 


/I 
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Chapter 6 
6.4 In equalion (6.2.14) (et 
as = Z pk) poylx)> i ; bly): wy) F 
then, with @= 4/(4-tp) we have: 
— ae rere es @ T" 


wy) 


Tz z(Z Z 9) pyle) i ‘] 1G 9)" 
= opfipe splalz (x. yeopns iG s ) y #4] f 


= exp Kek - -(i- *p) Eo(“ ff which is (6.2.45) 


In (6-212) let aty)a ert Oz t-tp 


and bCy) -[Z Egle! pyle | 


oe £. (Hp)| = 22 Zz, rep ybd Z ope vet 
<[z(Z q¢&) pyyix): “*) sap + twp z(z pyle) “|? 


ay 


Finally for (6213) let (with @» xp) 
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Oy)= Z 90x) PCY) amd by) = wey) "Fr , 
Then ot Ey (4,p)| = z(z ad wr ha Foy he ape 
e “P* L2(2 (Z 40x) poy) “ye [z Zz “9 a) 


=n ape - “p Fo(st) . E(6.2.11)] 


6.2 (o) Using, the Sawe wethod acin see, 641 we get 
E[ mx) | = Se 1 for the correct path 
ELm(xu)l < - B  ,for the incorrect poths. 


(b) (6.2.44 ) becomes r -«p 
exp} -E.Cap)} = z 2 900 py dal a / 


(6.2.12) ts unchanged and (6.2.13) hecomes 
ot ee 2. Pai ig 
sri nteple Sole fea oT] f 

omd (6.2.15) -6-2-11) becowe 

exp Eel < exp {ape limp) (HEE) $= 8 

me {Lease pe] f emp] reap) ap Mia) a 

a }-[Ees (4,@)- ply < exp ype -(i-ep) “hey i His) 

Skesy . 


S92 


cE) Ths ge 

For §,41 we meed fr < Leap (a) AG) ; 
vheye 6 

for 8: <4, va need P-#@< « f(t) 

oy K< [6+ £ (i) ; 

Choosima, a = 4/(44p) , this results iw 


oe aw] Re ptto(p) , and thus 


i+p 
Fe Cc>L) < ALT provided (4) is Salis fred . 
(d) Siomilarly an 2s Beda LOSS 4 
Provided (4) 45 sais fied - 
C2 _fesurae waned, =KLT, wad i 
(A) Ké)=2 = L[R(E>-4).@ c>1)] = ZR (E>) 
L=0 . 
“HZ < 00 provided | ibe ; 
This implies 2< B41) =o. 
(b) EC?) = ay LYR(E>t-1)- PCE>L) § = ZR) 
hizo 


ho 6 ) 
= KZ LIT <oo if: Pre ; 
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This implies e< fo(pY/2 =k 


h- 
ere) 2 FL"*Q(E>L)- KEL ee 
L=0 


if p>k awd this implies K< LibVh. 


64 saps rs i §.3 Lhat the block error 
~ protabilit tee coastel. enculcectional Code, of D branches 
a cow det Wk is 


R< D (ge. ‘ya 97 bk Fe Cre 
{- ade &CR/R 


Since, abeady slp we have b increase the muber of 
whobiows branch 2* the mouber 0 compuledions 

reared a We and of k bh $ is r bh 

22? 


by 2h, 236, ners a) Jar ae 2 


) a es 


Bites ae ve AD ae. 
Now let L. 2b ana lol , so we have 
Pubosen, eva" ie pent , of p< and 


D. O4-2°T }- 
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6510) kerk = CCP) —shich iples bert. Felp) 
K Ke)-p%elp) roy Lp kip) 


led « pou pk (l/in att) 
Ete) rE " Bolp) ~p £ Cp) 


{= 


bk, : : 
where L tekec the pace of 2° tn Sepuced J chcodiig 


(b) we moat add one constraint lenath LH, to kevit airs 
kevit represents the knath o 2rrop events C + 
ls} ovvor ) and Uwe ‘iss eos Goes = tele Ele. 


6.6 PCE.) is the block coding error probobi lity, Where 
there are 2i-4 diver In paths Oo f block Je f. ead 
over the ensemble . The re s ieee is independ ndent 
Of 20.0 of the diverging weed berate Hence, 


panei Mo ee 
PCE) ¢ (2 mit ( alee pas?) 4 

for fer ony pé Los 

a{Z swopynh?) 


Thus i+ 
Ce < z P(E) < z woe a a 


a 5 AS s)p are, oF ~ui- -1)Eo(p,4) 


19S- 


a ert? 


RQ . on Nt44) Fol p,9) Z [2fe ey { 
eo ~aCTeiJECp, 4) Fe e a “Rt, vy’ 


t=O 
ond thus 
— - OTH) Ey C s) ) - 
% < e@ F t where we Yequive fo tt, de 


This nto is equivalent te 
R= a 2 Eo lP, Z)- Foleo 
| a ue re: £) 
As long as R< Co Pod oe then there 
exists a p e(o ) 4) ade sed bred R< 2 p-2)/p 


awd Fol, % >o T his ee at for P m nde Yauge 
the bw eset &pouentially with T, 


6-7 (a) va lenow tok the aren | thee lity 7 fea 


Bocek wal yuie is to choose m aly ),but 
(ly) = Fiera 
py ch 7 
= pf] be) pt i) Taw 
TT wy) 
ond pltalm)= 4, (4 2 Hath) 
Sant is dependent t ee peyrieey: a DAC so 
yw to)= Ply lem) = TP PC) Xm II pla, gy (ty) 
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oe a com tril W- Max Way 
plz) = ie t PC Yukon) TT eCYauak’) te’) TT 4a) Than 


v 


WT wlyy} 
Noon. Ari D-poys 
_ Tw Di PeKue) TE Patt) 
AT w6yn) 
rhe “el Wr, 
Ty ay Pn lx.) mn) wCY al + ) 
i) 
me a) 
Tou es Pal Xue) thy TF PU pebtan 
TT wa) wl W(Yw) 


Moxiwiging Po] y) is eqaclent vane ty oa Piny)- 
ey bg poly) = log Thm + 2 Pen 


= 2 [hg Sido lM 


= L(w,y) 
b t ; AM ; 
Tha cc tec ai pee ye 


vs defived OS Be du/nhe . Se we have 


L(m,y) = z= [bg a + log ea 
= L(wy)< z [ont 2 | which is exactly (6.4.4). 


wlan) 
1947 


et 
‘Galiel ont om ate maine gar agree 


sie) Ke Ky lh), y= if< <®} cit be a a ee oo 


wr WOW condition the the right hand zl ow the wa rok : 
Unerased posit tows to apt 


iA f rie, OWES woh y in oll unerased posit 1DUS ; 
TL pose x % Ch) ogvees with y iw all Gee 
cositiols | Ydre ae YL duerased posix tows 

ite ove L uvercsed positinlss} 


Zita) (202) at 4) £(k-3) -2 


Bet wart [gs 0-4) 0 


: qed) _ the) eg G-4k) 


Let Boz ~ log Z(Z qfyeyis) 
x fey Z (LFF rq) 
abe ( 2 E414)e sal) 


a 
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Finally we have ~2U(k-})R 
leet ocx, x pe Al 1eo2 ) 


ie E+ 2 hed z Cx, xgelh 4) 


vlere 2(%,¥1:(1), 4) = oR {elx, sith, y= 2Of 
+A. wlelx 0), 2) 5 1} 


<lh ~2(k- m 
=[lU = ; = ra ie okie | Fs kj grep re 
) -“s - | hej Ce ; 
: z, Pifo-g) fe-abhe) sr 
_ £ie ed $ am) Gs) ako 
(ee ae) > 
For cowergwce ve need a a or Brite. 


(b) Cleawly the set of paths that reads the top of the slack 
before Ses avd agree. with x in all UmtreSed 
ra . a ave of all oe as which ane octen, 
~ eae sure of a the top of 
ty) 1 7 ¥ 

Si wh). j : tM ce ve x(k) and Foley 

wih y, fir —— hions 
af (Un) veaches ‘ep before x) | x kL) ogres 


pm al\ untrases oxitons! 4 


» 4% rene Wy) agrees with y im all F erated positions 
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The grolck lity {nek Sais) reaches the top of the 
Sek before %(k) Owen, c ORE with y um oll 
unerased ‘postions is 4/2 » because xXie(k) hos the 
Same wettic in all nodes as the path, x Ch), 
Using He vesult im port @) and this feck We gek 
; ~2(k~1) Re 
15,65: U, 2) ite + z d 


ind oS Ww park (a) we wll have 


awd fev Sheree a of the series vequires : 
R= > Z Ko 


X00 


C hapter da 


(~) 


= N 
Zhe Let PECviwy= TT Pala ond PROW=TT Pen), 


=- m=\ 


m=\ 
Nn (v\u) 


Then Tet ) 222 ani Prter|yy de BE Fy) 
N 


(m) i) 
=F > (U) Py ay} Z vhs Jn ee . 
ae PC Va) 


m=\ 


*( v\u) 
CP) = SZ Anlw Py viv) In nee 
ay Pr (Y) 


ok : pm uw) 
-— 2 is yEy ETS Ql pm (a[a) dn p™ ry ) 
at 


New, since ah ey 


= Teo te) er ee 


m=>\ 
Px (} B): Pa) 


= FT Qolw Pa (1) bn oR ww) 


eS anit uy £ Px Cul ¥) PW 
22 Qk rc f Be Fv) Pylwlu) — 


s  Qp(¥) Po” CY] 4) Pal Y) yes 
¥ Pa (v) 


2 1h) = £72.) : 


\) 
isM 


arr 
For amy PuCv(u), let PC val Un) be the marginal 


Comdrtiena] distribution for the nth Pair (vm) Un) 
derived from this avstributio . 


Note that - 
20]! 


x 2. Qu) Po(v]4) daly, v) 
es N 
= ZZ Qelw Pw yy Z 4 (ua, vad} 


W 


= { ZZ Qepolw deus} 


22 Q (4) {i . oa Pp (v|w)} A(u,V) 


Hene tf Parylu)e Pon , then 
A N ms 
Pculu) = ry p' (uly) e ee 
A\so trsm (7.2:46) and (7.2.41) , we aure 
Wey) > fe Teper) > 14 mE") I(P) 
arth equality — of ond oly if  Pylvlu) = TT p (Una) n) 
m=| : 
Hence Re(D) 2 RID) 
Sine for omy =P(ulu) € ioe we cam choose 
Pauly) = Pee Var Ur) @horg IC Py)= LP) ae alse 


ot Rul) = RED) 2 B12. 
ee \ \tf +P 
-E.(p,p)= b iP Poracuyy) FP = dy Z at 


\ \ 
[Qcajy)* a = Qulv)”"* Tse (Q(ulv) 


A Vite 
u)= = Pew) rau] = = Pry) Qlulv) F A Qculv) 


foci t = ott)’ toe olla) + (t€) - ot" u) 
yitr be | 


- + (v) Q(ulv 
= dtu) Jn alu) + 2 me a Qluly)/"? 


LOL 


Since 2 a(v|y) =, = p’(u|u) = [Zelwy =o ; sae 


[ant V = ottuy) Z pioqw be Guy 
u 
2 = [ atu? |” 


HES P) = Linc attuy'* |” = 


Z dla 

ar. ) 

2: madre 
“Zt mw “pe we P(v\u) 


EW): 
pap wu) B(v|¥) Ly Bru) 


/ 
[atu ** \” a [ut] 
Z ata" Z die"! y Z diay'*§ 


Wu) = 


P(v) 
W (u) | = alvin a5 sui . WC) 2.. B(v|y) de Par 


(vu bh, Be aris ed 


aly 


i} 


wa) } 2 


/ | t rs \ / 
B'cylu) = peu} Zw 1 Qtuly)/F)’ — Bay AW Qtuy"* } 


PW) Quy rF Pcv) 
| Ries ahaa - 3(vu) Yn rae 


B(v(w) aly 


H) 


ae 


Ivf ok (A) 

an Pv) 

it? Bui) | Lon B(v|w 
) 

= = Lwwaw\w) Ins = 22 wurpevt- soe 


pow 


— 2 aww ba oop ras — 


— EGP) 


(v) 


ZS Fw (wp(ulu) + tw) 2% |W} Le ars 


a W( Wa B “WvJu) 


oh PCy) 
ZZ aww) | Fe rae +E (P» iv) 


' 


(v) 


Pi) an 
+ ap be Sulu) a a nae y\) de pac evi 0 
= 1 p> Wu) al Wt bn pana * = rege == ota) {vibes io 


—[Eo'e,B) \? 203 


=inte -V<f<6 “ty secand> term tc mesartug ; 


: 
of) = tu pcuja) { be SEY 


For X=) we have the bound - (Ma x)” ay aX) 
So that for LX= {(wv) ; Pv) = @(vlu) 7 we hang 


| Pru) 
E, (PP) < Te. merase aul 


(v) 32 
os sai eb con 


ae 2, 
at a Taare Wie (vw) 5 ba 


Rr UME we hare the tmegual ty 


ot u) 
In Mu) < dn ou) — In Qualy) 4" = dn yt 
Pw) 


i. ya) ae 


- bgay choy 
Now 


Dy ltu) = We 4, btu“ -u5 ln Low (2) | ae 


Pru i 
(|W 


= reg | bn ta) — Eel p. PY} 
( ‘wy = (aj (Lawiw) + Ecce, P) —2 Evlp. PD, ow] 
= Ue)? (haw) + Grp, Pot 
Therefore we hawe oe 
- Ey'(p.B) z a +(qq) 3 = 134) (4nd) +& (7.24 
Tt % easy to show thot the unttorm distribution AKT Mi 206 
the first term ow the bracket so +het 


% v1 (Inwoow)” = (Let) =a) 
A\so 


Gnd 
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ft. 
~EolppPy= dn 2 (ZF Prod Quay") 


= Mn Ze 3 Pru) Qtuly))'** = In (Z Qtw""*} 
<n(Z') = dae 


Finally we then hue 


Bei = wep + (ee) 2 (WA) 
ime -LSpso — chsvse ce-s. Thom 


-Es(p-P) =< 2+ 1b Dod) eS 


7.4. 


Corollary 7.2.2.: Given €>0, thy 
tsts a block cde of rute 


R<= ROD)+eE with ANeVOpa dyshiotiom RCD) aN 
A(B) = Dre, provided thot D>Dewin. a 


(2) Define Ds =D-85 , 8>0. The 
There exists a cde B of rite R= RCDs) +8 = sith 
d(BY=Ds+$ =p. Fer any = >O — dkaosa § smmall omough 


so thot RCD-s)+t S = @pve 
Ths is possible 


Sinka Rio) ts Conti yous m D for D> Dmin. 


(b) We wn aluays find 5,5, >0 
thot satisties 


ao ymall as we please 


ROD) = RC D+) +82 
There exists 0 cele B er yee: OC RS RES +h 
sud thet A() = (D481) 45, =DreSrdy. 

Thus = fer amy €>0, tore eaxids ace Bf rh 


R=ROD) wrth average distrtion diB) <Dte . 


205 


sues 
~ Swppose in Fig. 7.6 we hoe 408.0) =D. Let Pycvaluy 
be the conditional probability of vA given 


Soura. Them 


uw oud «ot =the 


die@c)-E fdy(u.vad|B,C t = ZZ Oatu) Polvlw dAn(wy) <D 
Homce Cis efin and 


Sing RCd)=RCD) 
We haw 


( sea problem to) 

RCD) = Rn( dD) 4 <1(Pr) 

The dota proussing thesrem C Theorem |.2.1) gives 
TR) = Uy) = 1 2a 


— 


cate 
ae Nc (ee) 
Hemce Rib a. 
os 3 elo) se ten CB ol BE. 
te 


om Suppose a ev» sodtishies 


Ve Piv| a) 

R(D)= TCP) = Ze Qiw P(u\y) F Quy prvtw) 
A\so aioe thad pcv|u) 

C= ore az Qin) Pev|w = Gru) Plow? 


Now consider @ DMC with impo alphalact YU, wtp alphabat Ue, 
amd *tramsition probability Pcu(u) 


= 1(P) 


a0 Helloas , 
Pia ar DMC ‘ USor 
U P (via) eae 
Them the average distortim is 


ES Acwwy} = pus Qiu) Pivju) duu) =D 
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Sima Pe Py, Hence aithoit amy emcoding and decsding 
we adhieve average distrtimm D where R(p=C=R. 
Ear the eKtiprob able bigary sours with error distertton 
at fidelity D amd the binary symetric chamnel with crass aver 
probability € whe €=D, 
Ql=QU=Z, PCols)=Palid= OD, Plolr)=Pale) =D. 
d(u,V) = |— Sup 


Pcv|w) 
2, Qu’) PU|w? 


: 3 {2-0 1, 12 a 2D Ine | = (-p)dn(ED + DID + a2 


TCP )= 2 2 Qiw Poly) ye 


= Ln2-Hip) = c =RD) 
E tdiuwy? = va 2 Qiu Prvlw) Atv) = tx Dil =D ; 


a le Pacvlw) =f) 2 Y= vw 
2 veVWw 

Ohne NG) ei a He mani detain coda) —tledes 

fru, Here d(B)= FF QuwPa(W\wWdyliuy) =D 

so twt Pen . oe 

SM RDY=RVD) forall N (sae problem 72) then 


ROD) = Ry (b) < re 22) 
But in dhopler | we howe sham that 
LCPy) < Hy) = > Pon Ls < dum 


Hence 


M 
ROIS FZ Pri shoe 


| 
Pm N ” 
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Ue 
From problemn St lee hare 


—_~ 
Fe = 2 : = a 1,0) Bal, \ Lt (s,4)] 


Dong > 
Vis. g) = erg PE 4a) ( 1 Ply P| x? ) : 


$2920 
Les make, +e assumption s 3 


in My ae -% -\/<# —N/* 
hee "TE Mi COl ow eiiea 


Dhong S*ef-4o) amd {iw 0 for all ae. 


ls 


The optimizing “5% m 110 te low mites 8 alidh ts 
the case of trterest, Mosh channels st interest also have 


+x) Sh ter ol $6 : 


Noxt detine ~s 
Meg) = go gr) (Z LROWPIP) 
aay es 
= >> Un gin) SOG 
x 1! 
where 


din.n’) = —da( Z LpHw p> ) 

Note that in general F(s.g) < vig). Let ac a 
Lagrange “multiplior fo the cmstracial = flm=) and 
minimize — Jeg) = £05k g - 20 = 4) 


Subject +, Zim 20 tr all xe ond fixed st. 


Them 
oT (4) 


OE) 


* / 
= 227 gine” BGK) — 
x 
17s" 
with equality fey x cud ah Coro 


Rence 
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J K 
Fist, gh) = Zz Gin ine” HAR) 
= x 
: S* daa) 
iy ae Grn e fee a* (x!) 50 
= max + s*dcn,7’) 
x’ = Pme =(s.%) 
Gl’ )70 oe 


amd 4*(x) ts alse +he probability destributtoy thal 


minimizes Y(st,¢) . By assumptim Pn)P0 fer all 2EDL 
Thus * / 
i) = z a* (x) e° dns 47) all x EQ. 


Now cmstder a sounr with a\pha bak (GS ee probability 


| vow : ue}, representation alphabet Wax. ond distorttm 
diuv) , 
Define P*(y) = 4* iv) >0 ay he 


omd P*ufa) = Atw) prev) e289"? 


ong 
Aw) = = Pryes aww? - RICA wD 


This daica of = p®(viuy_— sotitsties the conditims of 
RNERTO TR nee ow poet 


Dy = > > Pw Fe eaenoe 


x’ ¥ (s*, gD A489 
RCDst:G*) = stDse — dn 11% §*) 
Now recall thot 
ExR)= wy - t({R+ LES, 4) | = EL Rt dd tf] 


oY 


R= s* Ex(R) —In¥ (8% 5") 
Hou = ExCR) = Ds whe R=RCDsAif* ). Hore by 
symmetry ieee aad R (DIR) = RID 4*) ok D=6,(R). 
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In general iF we velox the umd Tt 20: fer 
all ~eX_> ten 


AKO 
Vis < Faye? 
S5 me 4" in) e 
with equality chen Fin) >o | 
Them defining P*y) and P*cvju) ao befere with 

ay = pe Pyyesdiuw) < Vist, 
rusult, ™ 

RCD», ¥) = s*¥Do + 2 $(u) Indu) < SDs -Lh ist, @) 
a 7 

hem using Ex(R)= eT R+b Vist, #)] 


wohonre 

R=RC De, ¥*) 
yields \ 
Ex(R)= %& CD ROD, oF) 410s, oy | 


> EL sD - Lt gh) thar] = Oe 
Home Ex(R) >D Dhonr2 
R=R(p) = iii R(D>4)= RDF) 


In jeneral , Necessary and sufficient cmditims fer 


ews aks 
VCsh GE) = 2 Fn) eF dH) tar oll a eX. 


Ge 
With — dx979 = —ba( ZAPHTMPYIKD ) — define for 


code Caf mrs-kwt of bodklowath Nand rate Re AM 


Pg 


For all such codes let 

dON,R) = MX dma(C) 
let (" bea code of block eng th N and rate R thot 
Gchieres the montmum minimum distance with the fewest 
codeword pairs. Hemce 


ACN Ry det) = gC >) dal x oy, 


where : 
SOM\G Ja in do) 
xe 


rs filcas fon on tt there entsh an 
ety ok ir dCN IC) > dimalo them 
by inter chamg ing Re th a cdeumd oe tha 
achieves the minimum Atstonce ahem paired wrth ansthur 


cedeword, there uruld result a new code with he user 
Pair of cedeurrds thot adhieve the mint mun Acstamce . 
This contradicts the defintttm of T*, Nao we hwe Hu 


Gilbert Bound : d(NR) 2D where R=RID | 


ee ee 


alc) = = ty ACXIC*) < dann) = dCNR) 
The converse saw coding thecrem states that any 


ade with awvevage duckorttm 40 C*) must have rh 
R > Rl dle*)) 


For D satisfying R=R(D) we then hare 
R=R(p) = RC ACe*)) 

or aes 

Thensfure d(N, RD > die*) >D. 


2/1 [ 


Teo 


Define D*= lim. inf EY dcuttyo$ suck a limt 


(a) 

exists forall 4 O< EX ACU Tat < do < oe. 

let evo axbi trary | Then there exists a value Le 
Such thet yoy =e 


Let ft) be a mumber” saty stying 
Fid= L-kM, oc fede, +: on integer. 


Then for orger L, 
+ 
Efdalte} < 2 Epa tedy + Ef acuta)? OH 


= Ae (yee) ad 3. do < D*+e + se 


oe on ot Ex d(y|Tat < Dre . 


Sine © arbitrary - Ye am choase € vey small , 
dCulTy} = D* 
tists wmgusly and ts independent sf source outwit sequence, 
For a fixed 2, define p= et EY Ac ul Tad? 
ae: 7M > 20 Aor “Tm ts equi valent ft.  \r% ter 1a; 
both limits shold be same. Fer arbitrmy €/70, thre 
wists a walue Mo suk thot Ef dul Tamed) <pr+e, 
ers A im) = /m— ¥ WM : o< 4%tm) < Mo _ 4’ sam wmieqor . 
Them resk of oxyuments are some amd finally we howe 
Gt) Pr 3 d(u\ Toth) >xb = Pri dCu\Ta)+d(ulTe) 2xt | Varo. 
“0 
ES dC ul Tet} = Jo Prt deal Tat.) 2at da 
mo oO 
=)" PF dlulTaedultadeatda = Ef dcultao+ dul tat, 


AILZ 


(b> PY dgiulvy) < D+S | ut = ri = dCUs Va) = D8 | WY 
bet dls ValeEa. dhon . 3's. are iad. fv. 

ond Pref x=0F = |-D, Pri x=l}=D and =O k =D. 

Hence we cam use Chemot bound. Assume pene 
Pia = tea it) = feo eo 


ohere Av is giwen by Gene = He? 


So hey « Pr} x=0}-0: OM? + Prf x=3-1- e728! = D¢ 
Pr %=03- 27° 4 Pp Aza}. or! \-D + He” 
a BES (D+8) (\-DD 
Gee ee oo 
py enh ae ip Oe (0+8) U-D) = \I-D 
e \ a = Cl rere =e 
\ 
- = 
= 
ne | oa 
le (pes (ete a 
oe. 1 (xOCED (D+8) C1-D) 
og D(C (-p-3) + 8 ds P(1-D-8) 


(D+3)C iD) : ne a 
— itd — \ ae ae f 
S Le D(\I-b-») th D 5 nes 2$hs =-2R(D) 


Sina =RiD)= urd -¥(D) = Jnr + dDkadD + CHD I(eD) 


od RCD) = per —dncep—-l = be 
(D4) ED) (p+3) (\-D)" 
1 Sse = De ee DJ.D — D dn (\-d) 


= bn (PEBY + Jn( Ue)” + DAD —Dbc-D) 


FS wrt dn (1D) + DD -—DInc-D) = ROD) 
ie -r. (D Y 4 te f 
"ie 2d fae mb [RD)- RD) S| 


—~ni CRODD-R(DS J 
* Pr ¥ detuyw< Dt3| a} = (i- ager e : 


413 


EC2iJ=E[ # ef paths with drstotion Dtd o less from ul 
= (# of total tree paths) x Pr} a pth has distortion <D+t3 | 4} 


=2'- Bf adsUwe Ded \u} 
) enn CR-R(p) +8 RD) | 


cas ( }- ad 52 
for small 3 such tut er>RID-3R(D), we cam Find 
\ large amoush +. how og BE 


(> From “<*the branching proass etinction thesrem , we knna 
when ELA >i, Pri agret SP[2arro} and 


Yow Pee mar 


twa >On 


Sma Zm70 wmplies Hot tore exishs at leout UR 
code urard -v =(W,Va,- vm) im the tree suck that 
d(u;, v5) = pas FO ON ae 


We haw Pri 2m rot < Pr} dcusy 


3 %3 


4 P48 WG =). my 
Or PL am? Ot < Pf LZ 4CU5.U5) SDH8Y S Prf d (ul Tmt) < 48} 
Vien Gr] dul Tmo) < Ded} > him Prfanro} =4 >o 
@) For arbtoy Foo, choose L large omayh so that 
D*-3 < Ef dcultyy <d*48. 


The expectation taken sver the tree code msomble tor 
wA Sour oud put oe U alsa con ba considered Go 


fe ‘ 
Ej d(ulT)} = He z & dy Cuz| Ty") 


Therefore 
Hm Pe Aca) Tmt) £034 = bem Pf am Ze Ml evr 3] 


™ 
= Pet lm ae 2 delusl Te) S D*-8 | 
= Pr} E(dcul Te) < D*-8 i =} 


R14 


€) From ©), lim Pr} dculTme) <D4Bt 70 . for amy §>0 , 
amd From A), wn Prf dtu Tl) <prg'}=0 . fer omy %/>0, 
Thredwre , forall 370,89 >0, DtS > _—S" 
or fer amy € = 848! >0 , D*< D+e 
sts R= +t dno sotsfies R>RID) , there exists a 
Sequence of tree odes such thal 

SR EfAcwt <p ow hm Ef dew Ta} SD. 
Glalk) = Prf divlte) 2 A = Pr{ rdw, deur) =r} 

Sina Gslh) does nok depand on soura ensemble. 

SAAD = Pref min Emd-dcourl a4 = Pri dent) 244 


~_ 


whore do) \ ng. do, v0} 


VEUnd 
} 
\ 
3p etters vet (0) Te 
ey me 
Gee eS ‘CReaSeenaaoed 
“i oe 
(a) 
AS 3 min (2) 
— 
caf ed ee xk 
— eT 
je— --—-__---__ fx) bromches — - A brome ue oe 


Define A>= # of "77, wm SSR Poth . jatera , 
Pah) = Pref ds=#} = (4)" GG) 
Then — din (del) = min , 13 + dni wl) . dnd , since 35 and 


jaler 


Oa fA) ane 8 ROS respectively , 


GHIA) = Pr¥ damn (M41) ZH} = Pr{ 5+ dw? (4) ZH, 5=terrf 


ax 


Q(a| del) = [rf Av damatd) S53 1 =|Pe{ dwn WD zt-a}] 
=| Zag Path Pr{ dannll) = % -a | a=k} |” 


LE GPU ee dant tw 


p= CO ool 


On tor 4-0. 
Prf dc ult )=2} = (sy 4 (mys a(R (3) Ups | “I 


GAM)= RA duity 22} = Prd duttw=a} + Pej pet 


2 2 


ie 
tot teldanta 3] eae Lies “%) 
a) 


if 


2 | 7 i G(4-4]o)| 


zee! : : 
uomditim om the distortion : Qta) d(woydusdo ~ x>o 


With the same weg wen on page 424 , we hore | G¢-(1-5.3)) 
dtB) <= OCP) + yo” Qyl 4) Pa(viv) dcul@d Siwy; Bd) dudy. 
Using Holder's inequality and S°=B  BHo 
FJ Qacy Prd dj) & cys Bd dudv 
20 Vox 
= L {=| ApwPy yw Ldap y du dy | 


x f ee An(wWPyMIY) ScuVsB)du dy | ee 
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[ef aww Poly) Fd Cu|Bd dady 


= fey Qn) [dule[du < ce Aw Ldiwol" da 


= Ps" gow [ Edius 0) du < (= awl Z duc] 


~ ea | | aus a uzo)dus | = 1. 


erie = 
Mm JB) = DEE) + (eM [ {2-]_aowratula & iy :@)dudy] * 


-~ 


Now averaging this over the code and using Jensen's Inepuality 
Dields 


UB) = OCP) + dof || Qa Pew SCavB) du dv] * 
= DP) +d, Q aLN ECRs GP VI 


Define E(R,D)= sup mox E(R5p,P) , ond sina D(P)<D 


Pep, A<pso 


Wey epi et 
aT Le] 


There exists a block cade with UNE ORL distotion d(B) 
Satistying d( BD = D Hace —TeA-V/r IN E(R,D) 


ean E(R Disco fa RoR ip 


L.15 We already know thot 


tunctio of D, wd (b) RCD) 
s+ 2D tor Dmin SD < Dmx. 


Want to shoo that tor De C Dann 5 Dnox ) 
ED) is 


@) RCD) tT a AMINE RASI NY 


Ss a cumnx VU 4tmctim 


umtinuous wd >) strictly decreasing . 


1) Suppose RID) is discontinuous of Pye (Denn. Dimon). 


Then tor some 8= 10,11 aml ster sume cmall a >o, 
such that 


Dam < D,-*x <Dyt+e < Dmog- 


Al7 


the tollowing Meguality does med held: 
RL ODD) + (FOCD A] < ORLY + (-OROD tL] 

This contradicts te usnvertty undittm ). 
>) Suppose RCD) ts mst stridly decreasing. Them thers 
exist some D2 omd Da such that RD =R(Dad, 
Din <Di < Da < Dmax. But this again umbrn ards to 
cmwrity umdrtio (b). Sina tor some @€ 0,11 omd 
for some smal] pO, Dmin = Da-3 <Da<Da< Daté < Donox 
the Following Me qualyty does not hold: 

R68 (D.-3) + (18) (Date < 8 Ri Da-@) * (1-6) RUDat8) 
Therfore the usnverity sf RD), tegather with +he prputy 
of non-inoreasing , implies that Rid) is a cartinueus >trdtly 
decreasing tunctim st D fer Dmin <D< Davox - 


Let Rid) = IC P*)= me TCP). Suppo Dep*)=D <D 
——P 
Then from the properly of strictly decreasing Rid) > RID . 


But when we sh Vo’ =f Povws DCPISZED, Mey , 
wd ROD) = Me ee = Cy aR). 


LE)’ 
Therefore we howe centradictio, 
If Pefe yiells RIDD=TCP). then D(PI=D. 


7.\4 

aoee lat) Ge = aly TOR). DceP*)=D +e dav) 
-Vp 

ane f=} Piviuds DCPI=SD}?.~ 

Define another distorttm measure dcun)= acuv)— min deu,y) . 


Them BCP) = ZZ Qu) Prv[W dca) 
= zz Qiw Pulwdcuv) — ZZ Qiu) Piv{u) pee. d(u,w) 


DCP) — ZQw) MM dcuw) = DCPI-Damn. 
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Sp) = ma Me Py 
R (bye ( Pe he r 


where F=f pcvlu) > Bip» =pt 
=A Fw S DOP) = 6+ unl - Posen. 


1D - min =- min 308) 
RED) pe ti P) Pe PotDmn 


\ 


RCD+ Dmin) 


Y 


ee = Fas 
TIS Dyag=0, Daag = mie Sf dconredenih = min as 3 


By the symmetry of source probabity und distortion 
measure , we how Pood=PQ). 
Suppose Pod = Pc = 0. Then P22) and 
DC P>= => Qiw Pcv(w) dtu) = & | 
This meams Po) =Pc1) >o ter 6 < D < et. 
Suppos?  Pood= Pons Sy PlaysO. = Then fire 
PGla) = Nw Pw) e Satu) 


we hare > Poo) = § Ao) + ACE*} Pro) 
2PQ) = f No e* + AUS Pa) 
Ss 
amd 2 a oc 
Ais) = A) = Ter. , Ps a i 


~ Rib) = sD + a:b. In(7eesD = DnB. + Ino + In (1d) 
= And + DID + (1D) dnCHD) = In2-H(D) 
ThE De De clon De 4G tke ponk at ahtch 
P(A) becsmes greater thon 0. 
Buk whn X&>5 , thre is mo suck a pint om the 


regio 0,5 2Dmonl | 


‘< RED) = InaeX(p) oz pet , when oe 


———. 


aly 


On the other hand, whow o<x<t, there exrts such 
a powt tw the region (CO, A=Dman) . 


For the regim D™epe+ ae, Plo)=PCI9>0 and PC2)>0, 
amd we Wore amother exuotim ter A ; 

=) = NS Oe fa) > ie 
Together with previous result 5 2= A(\+ >) 
we ctm find unique sdlutim of S and Dd. 

* ROD) =sD + at- nds = SD-sA = SOD-A)_ reDentdd | 

where 3 cam ba calwlated from Follooing egunsrion 

1+ @> —ne°* =0 
amd p* can be calulated from the omtinuity andrtin 
of Spe ay CD) 


aes Dr x e* 
a Co ee ; 


\ 
Sis From the result of the example TH Pager 442, 


we hore lower bond to RCD) 

Rcd) 2 H(UW) -H0D) — pd (ARN) 

tor ol Dy, O0=Dmn £ DS Dan = \1- mor Qlu) 
And we kno whn Qiw=VA wuell, 

RCD) = H(UY) - K00) -D WC A-D 
amd \— mox Qiu) = 1- A = (A-) & = (AA) MH" QWU) 
Buk in cose that Qtw is not wnrterm, this laver 
bound gives the enach expression ter RCD) oly when 
for all vel, PtW>0, and the exist s<O and 


AL20 


Muy, YRhayA which satisfy the Fell swing megaalrty 


‘ . 2 A 
with eguality - = Aw Qu) est)? = | 
uU=\ 


The choice D I-D 
S= a Mu) = —— | U=l.zae-, 
Ln CAA CID) * Qu) ok 


Sotisfies the Tmegualrty with eguality. Avd rom (7.6.31) 


ole eo) S sdtuv) _ ic 
Ti | aes 2- P(v) e = 2. Prvje™ + 2 P(v) 


A é 
= 2. Pre* + PlwCi-e®) = e7+ Piwdi-e) | suell. 


\ Qlu) = ea 
or aay ee 7-0 ~ @AXED) _ A-)QlwW-b 
ee i D ACD) 1 


(A) Ci-D) 
For AZ DW < (AAD in Qiu), the awmerntor Ts aluays 


Positive . Sinn A668 ot Quy < x : 
D< (aay Qu) < (aay = \-! 
2 ACHD)-1 > ACH-14+4)-1 =o. 
So thet Plu >o tu, P<(ADQAW FU, or 
D< (AA) @ Qty) . 
Therefere for 02 D< (AAD ™n QW , 


R(D) = H(U)- Kop) — Dwar). 


Because of continw ty vo RID) fer OLDE Dmx, 
ak =D=a (AA) min Q(u) | the equality also satreties . 
2.47 SOU d(u,v) User 
*  aly=QL)= QB)=@H*Q= Fd 
From the symmetry of souwrae 
ond drctortim measure , we knad 
Pca) =PC2 = Pr2=P(a) & Pa 
Pisd=P(4)4 Py . otherwise d(UN)=c86 


22! 


* \ 
Dain =O, Parag = min 2 diuddus) = 2 4 d(ut.=3 . 
Let's think about some special cases first . 


O) fae0  PE-G). Fla -\. 
Then = Pcilu)= PCalu)e- =PCCLW=0, Peatu) al. vue fir3.4} 
DE ZZ Qw)Pufu)dtuy) = TS Z-Pcyu) dew) = 3 = Oman. 


Rb RO” 
(b) Paz - DVL 0. lo. Then PCulu)= \. Pivjuj)=O 0 V Fu. 
= ak RCO) = RU Dmin) = WU) = nd = 2m . 


(© Paso, Pb=+, Piq)=0. 
thon = P(5|1) =P(s[2) = Pola) =P (b=) > stherwise § Pilw =o. 


P= 4xtx\x\ are 
\ 
Ru) =1(P) = re Piv|u) dn _ = 4x4xieln =i 


GQ) eas. 0< Paka Sard 
Again by symmetry , AQ) =A(2) = AB) = ALH =D. 
From Pufu) = AlW Py) eS ACU? poe om 
Paw) = = Qt) Pcvly) = %APW 2 e sdiwv) 
Henge, we howe 
Paz tAPal = P= xr-P,-2€° 
De ee ae Se 
* Ds = ZS Alw) Qty) Piv) eA deyiy) = Pisp+P() = > Py 
sDst ZQwlrAW = -Inr- 0s +44 -det 
22d -Ds'dn2 
. R(b)=ahi-Dbr » 0<D<\. 


R( Ds) 


)} 


©) Peso... Osi ts. <o< PiD<|_ 


Erm PCy) = 3-dPv) = Stn? , Pi) #0, we hor 


£2LZ 


P= aR.2e*, Pad= g>Pa). 4a" 
boomais ee a ae ek 
Gl - Wi 2 Pare 2 giyy) = Bi4 PL e> + 4Pi7)3: 27*4 
=2 Ph +3Peq) = I-P43 PD = 14S PQ) 
R(CDs) = sDs + 4: Edm 23 Ce ea ae 
= G-Ps) $b. 
' Rib) =(o-Dia io 14D 
Therefore the rate diestartio functim & gwen ao belao: 


pRB) 
aln2 9) 
| (a) 
; | ee 
oO 1 2 3 ir 


Wee 
For SOme <0, Sugpose 3 Ps(v) . 02 <u geo? satisties 


XG 4 
(2(u) = eS rf.) ec (sy? dv 


— = AC 
jee € Az 


and chose = sudk that 


[rw] = Qt ye esa 2») da 


Then this dui st Pil) amd A satisfies the necessary amd 
sufficient omdstims ef Thesrem 7.7.1. 


dwu-v) 
4 -~09 wwe> dv eo U 2 i 
[Aw] = ie Ps tae. -{ ress ” hy 


(a os de da ‘4 ee f 
if Pv) >0 , P(ulu) = lw) Ps(v) esd un) 
es Ps(vjuydy = |} = Au) Be Ps (vu) e Sdiu- -v) 7 
2 ae atek, 20 d(u-v) 
eo s — 
i Atw) Qiu) "chealia Y a hae du 


a eSAl® Jo . 
423 


(ames? a3 


Hence, 


ds =| Aww) Qu) peste diuv)dudv = 
ea © 6 sd) 4) 


—_—*~ 


amd RiDs)= 8 Ds + (" Qw LaAtw) du 
= 5D, + G76 pa ie jetta = Rip Ds) 


=_ 


Shannm's \swer bound ts glk. 


7 \9 00 
(a) R(p) = h(U) +sD- he est d2 & Rip (D5) 
20 
diwnert'® da 0 
& Rint D, Ss) = D— Jam 8 5 = D-| da) (2) da 
ie Pts aay We ee 
sda) 
hong a ad 
g Gs(2) 8 2 osha 4 
ae ee die) 
2 a). wer ent | dae de 
se Asl2) = a - — 
Ss (eas i | ‘2 esd) eae 


= dea) G@stz) ~ Grsta):(~ dee) ay) as 
2.Rip (Ds) = - a d(2) ®@yada 
20 2 = 
= ae [diy Gstada + [ i‘ dia @sta) de | £6 


~ Rig (D.s> is umng O fimetim of s. 
A WAAR — movRtmuUrn at the value of s thot sabishies 


Po 
= Rip(Ds) =O Re. D=|  dtwGuade 


(b) Ny se ) dca) Gta) da Park We want +e. shmo 
Rip (db) = h(U)- ecg = h(U) + 20 L- hCG) ; 


iS 
Wont to minimize 3; —h(G&) 
subject to 5 Gl@2zo, [Glode=i, \dieyQapde =D. 
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amd possesses 


J(@34,5) = -hCG) - nd f Glade — sf dew Giada 
Gla) 


= Sara} + WG) -du ol ~s dea} da . \ &(2)> mw — 9 Sd(2) 


Toke a variatim © 92 about Gra) such a 
tz, fe oaid J @de-o. 


wee e-7N(2) ZO 
Gla) +€ @) 4 
Them DC Grey 3,8) = | [awreto] dn Siw 4 


= Jae bn Sas iis da + aw Sree 


d2 


2 JCG+e4 j8)) 


€=0 


-fawh—S a6 Qa 0 | 


at kK = a i an oo Sunctim of 2, thon 


= oe G+EN, od, 5. =. ond G(2)= K a psd) 
\ 
From \= { Girda = Ka See da, we hae d= K (ese 
Thhefere ‘ ; 
men Thee G (2) In G2) d2 
oe al J 
amin (acarl Inka +sd@] da 
Ge 
- 0 Yn jacode | 
Pe {die @cerda +| era! 


sP- bb We oda 
. sata) 
Rin(d)= h(U) + gegyl-hG@d] = hw +sD- he le da , 


I 


20 d cu) = dcu-v) =\u-\] 


a = rel \2 oy ie 
(a) es de), ec "is . a|) es? ee > = 


—Oo 


sd(a) 


= _ \st sie} 
Gslz) = f esd a a e 


20 Ae 0 \ 
D=| darasa da =| ia. Se have = BPP hear 


—% - \s\ 


Rig(D) = hi 20) +sD-h ie ede 


= hil)-\ -.(2D) = WU) -ntred) 
(b) When Por) satisfy (7.7.46) and Gea) = eM/ ( esd 4, 
dtu-v) sd(u-v) 
Jpmes dv ae 
Qu) = =1V(0) dv = Pow G,cuv) du 
(ene ie fers, ) 4 


let H(w) he the Fourier tranctermatim of Hod. 
Then we hore Olay= = P(e)-GsCa) or Pt) = Giw) / Gsrw) 
And G.(u) = a , aslae —jwa vee _ \s\ a0 er 3ae | : s* 


) 26 eo 
: ~ tre ~ NX 
wey as +e = Qu) + Dw QV) 
Taking inverse Fourier trmstom , we Wore 


EO) = 0) )-O oe KN SO. 


V4 


" 3 fe 
(C) Qiu) = 7 os) -%<Uucom, Qiy)= = rao ae RICU) . 


Sine Qiv) does not hare ys derivatives at v=0, we cam put 
tt as a delta fumcttm. Bit |Pwrdv-=1, JQwdv=o, D#O, 
judy =O =o + 2 We hove R= - 2, 
Therefore Pv) = Qw)-D*Q"v) = 4 en v4 x Os Sw 
Hema fe all> 0204 Dae = 2 bio A ee 
hi) = -f. Qu) dn] Ser A"] dy = Wn(4) + a IF ul Quwydy 
= W(4>+) = 44 (@) 
~ R(D)= Rip(d) = hivl)-Drlaed) = -In(&d) tr 0c DS 4 
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(d) Qiu) = (\+v2) , Qty) = = (-2)(2v) (i4v2) > 
Q’tw) = 2% (4) Ciav9y ao v2 cay * 


- 2. 2 = eu 2. 
Se i ie a 


Piv) =Qw)-D7Q"v) = # « iru) 8 (tue) + 407C1-5.v2)) 


a> I ~4 . 
att 4 VteaV Ci op*) +14 4034 


~ x > | 
= Ratt ff vrscwopyy” + 24 0? x I- = mit 25 
2 24 oe ‘4 


ht) = - (CF csw dls Cruzy7ldu = Piga) -2 
Rig D) = RCD) = 3,82) —2.-dn(2eD) = In (42+) -3 fo ove € | 


mde a 
~ Shamnen's lower bound ts tght tt amd oy if Sor soma 
Sco, a probability density Ps(-) satisfies (7.7.46) 


esdiu-v) 


Dhons Gm) = e 4c) / (Bee 


Cromer'5 theorem Says thot Q(-) ts @ Gayssian density 


iF amd oly it Ps) amd Gs) are beth Gaussian demsity. 
But ct d(uv) =dcu-v) # (u-y)” , thom Gs) cam mut be 


aA Causstan density , ond so thine does mab exist 
a PeGCyY hel “salisites (77 40) |. 


Therefore, tr a Mm ENNGY Y less  Gourssion SOUL and 
diusw) # (u-v)? | R¢p) > Rig (dD) fe- all D. 
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7.22 


d(uv) =dtu-v) = \u-vl fs ‘ d(2)= lam 


boa) 20 of fo, 0) ol | ans 
sd(® s\2\ —lr2) a 
[ce d2 = a < d2 = a ec da Y= ae 


ae 
oS ES) = a) 
dca) 
Gs(z) at e" d = aise : Sle 
fede = 2 PCR) 
<] a 2a am 
Ds = ( ps pote a Ge d 
: J deaecayda oy \ ze°" da 
pee Ge Be 
Pe <i Ck ee 


Rin( Ds) = h(K Sainee oF 4043) 

a ae = 

hw) ~ > — bn [Sax Leo 
ee) = P| EC) = BG) \. 
lag =|] = h(X) -LrQeD) . 
42: D(iD)=DD=, se) -Te 
a 
Ds= Sig - Rew(Ds) = Wu) - = era “Ja | = Wiwy—4 dn (s2eb) 


Ds = a » Rip (Ds) = h(u)-|- 


ie a 
Lemma: A lover bow to +\e rate distation functio 
for a continuous memoryless soure ath differential 
entropy WU) ws given by 
RCD) = Rigid) = h(YU) +sD = bol 
when uU*  sotisties 


pv") 
esau du 


and & satisfies the cmstiant ; 


Be we dew e edu /| aN ia 


CPiest) Clovsa Cu) = such thet 
fat" = in p esdiu™) Jy 


then ,% 
_ Aw Qi esau”? Ace 


(2 esdu) 
(ee = 
Henc ref, whers ie ts defined in (Tah) . 
And for ow S=0 Ure Voune 

Rob) = ope [sD+ J @uu be dtw du | Cas) 


ey Rip) = =p + (aw Ld du 
e: sb-| awl b. Qu)+ Oe aera du 
=31)4 hi) = au ee = RialD) , 


20 
~ Q( — be Ors diu.vt esau? du eae 
era 


Po 


Their =< Seturies tL unstraint - 


eo diwv*) 20 NA) 
be ie dcu,v*) du / Gee du QED.) 


Tf duw)=du-v) ~ +n 


(ert) fy : ( o Satu) da (" 0 sae) Ne 


_-O 


_ 
_— 


J We hwe Shamnom's laser bound - 
RW) = Riel(dd = sD +h) =i) er=) da 


ae) 


A249 


1.24 


—__— ~S- 


We can prove Theorem Fee ie | by proving simler 
thesreon to Thevem 7.6.1 firsk: 


CPs) = We Qiu) Prviu) dn edhe dudu &.4.12) 


XI re 


Theorem 7.6.1':3 A necassary and sufficienk  oumdytim fr 
}Pevlud} to ammimize TCR:,8), subject te ay inegaality 
eenstraink (7.74.4), ts thot A sabisfes 7.7) ond (7.7.8). 


The proof st Theorem A ees enactly samne ao that of 
Thecrem 16.1 with only excaptten that cummattm es 
changed +o integ rol . 


‘To find the necssary amd sufficient osnditims fr PER 
thot yield RD). from (1-75) and (7.7.72) , 


| = je Pcv|w) du = Aw) {_ Pi) es 4) dy 
Heme we howe (7-7.9) - 


By substituting (14D inte (1.7.4) and using the fact 
thot RIDE ICE) ulon D=DCP) > we obtain G10) and 1D, 


qa 
() Similar te theorem 77.2: 


RCD) = ek [sD+ > Qw dn du) | 


whore L\s = {Aw) + Ew Qiw 2° mae oe ved } 
Choose nN sudkk Ahet 

A(0.) Qa) = oe , Aw) Qw) = 
Them 


t>e> * 


£30 


-v\ 


esv \-e5 v-l pes 
srt) sWW-¥) 
ite? \xe> Le 2, € Fe 
i ot! es \-e SUD Vee G er M4 
l+e> I+e> | \-e+ {ee ic0S- 5 = | 
dia cuk doin of A ie te iS | Ghen 
(w dota) = iS = 


= = Qw In gr ee ~~ oe 
= WC) —tnCites) EI -Giolti-eD 
RCD) = Rip(D) = at sD + Hed) — dncites) +0 1-Qo) dn (1-€5} 


ma ms) = SsD+hH(W —dy (re) +L i-Qoldn(-e%) ts 
strictly emuex (\ af s 7 


e 


2 a e> 
Ee eer ng (8 Qa) Ts ey 
and we get  lswer boul of R(D) paramere reall by 


es e> 
Ds = wes | Lip CO a 


amd 
RibtD.) = Hii). oD. Madiee) +] 1- QW] dn (\-e£) y 


(b) tt suftian to show tha puramaler S ww the 
slope of Rigtp) of D=D. 


/ 
R = d Rua (D) _ ARipiD) ds 
Lp (D.) i a re 1S 


2] 
ab 


={Dets. aie [es 


a6, 90 Us 


io to 


With similar anrgwm ek 4. theorem TI amd Lemma) | 
AZ 


slu-v| 
R(Ds)= S0s* FQwiaMw , Os= TT AWOWPWYE Jus). 


hans [rcs = = Presta! vued 
v=0 


amd R/C, = IRD = 
See a5) 1 D=De 


Rin(D) = RCD) 


“Py 


: See alan Eltad. 


Soe Van aad (ao tlot 


(4) G@tem Yen an emr ts made th for 
some wu en hong Qa #4 , we haw 4.@) =f 


() Por O¢fe| . using Jemsom’s tn-equals ty , we how 


; , ~ ue] f 
Pefdeyu} 2 = Qu [ 2 Pewdld) auatyl 


on YP (ult) = Pr} Fed) = Few) -+ | Therefure 


\ a 5 
Pr7#Qewo< mt = Quw"* | ati) | 


< wt (ZQuw™)( Zane)! 


= ut (Taww/) A 


! 
HF 
shane Es(R:p) = pR- Eolp) - Ef) = (4p) dn ( Z Ql ) 


Note +hot Esp) = - a Qlusp) dn Qt u:p) 
u) Vit? 

dharg Qtu:p) ‘ Q(y) = 
2 Quy’) J 


Them Eo P| =W(U) , ad 


E(e) = Ww ae Qaze)| nQcue) - Quip) Qtu':p)| >o 


Thus we howe Eale) PR 


PH) 


ry Tad 2 
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Here EcR)ys both E<(Rif) 70. fer RP HI2C) 


. There exists at leash me encoder fer each N— thet 
satisfies this averaged bound. 


7.24 


Here redetin 20+) oe (wu G|Qv) ae oe 
; ULV u 


Pr} 4 zu | wy} <[%, ¥(u.G|$) cu wav]! 
Se nesn = FZ Quiuy)| Fe wey, Ulf) O (Gla UIQ, vf 


=F Ww FQuug*[Z pel acaw |" 


G#u 


= eo 
Prya #ul < 2 Que Qy uw Fab yl **| 
=m X Quty) |= Qulul yey 


= ef RYT = Quy T Quite} | 


= e ~NEscR:p) 
volono es 
Es(Q:p)=pR-Eslp) , Eup) = taf Faw Zu] | 
Note that 
Ese | a —Z Qw)} = Qruly) InQtyv)} = H(U|V) 
Heme defining 


E.(p) & mon Es (Rie) 
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—NEs(R) 


vor hase Pej a¢u} <e 


whine E.(R) >o fe R>HCUIV). 
Therefere tor amy N there 2axtsts am encoder Suncttm 


Po Prjdpu\ < piaeuy <e NS 


1.30: 
(4) The minimum average error decision rule is defined ley the 


resins: for ™M=|.2.---M 
L\im = £2 Om Pr l¥ Xm) > Qiu? Pw elain) fer oll an! acm 
x 


mck SO. Then 


ae f 
ai Paly|&m) [ fa evil Prsl yan" | 
sver the csde ensomblke and UST AS +ho 
gives Ha bound : 


2 ae SE | 
Pe-=< 2 2 Gy) Pol ylxy = an) pucgla | 


23S 


\ 


N 
Sine ae a < ae aa d 


™ #™ on! =\ 


oS h 
Pe < eae 5 ( Paya ) 


loz Q my x & au polyorr) 
m=\ ¥ x 


- = Glo ho gn) Fz energy te) ]" 
: e ~N Eo PA) hana Golf.) = z (z gon ptybo 6) 
a On. T ete 
(3, ac) =( Zag)" “(2 rt aca) ) 
= (2 Come i orb Este) 
ohare Esip) = (tre) dn} = QuyFr} 
Pe —NEae.g) +L Est) 


te = 


A 
Este)= b( 2 Qi) =bi=o, 
\ 
= Qi Lau 


= i 
A ee 
= —2 Qup) Quy) 
ee a 
Q ( S y= Qe). A=\,2 a 
acf Fa) * 
Es 
TT me Gite) thom a 7 0 for all 7 


Alse QI> >| v= QD) ind so Este], = KU) ” 
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(4) d= LN. Than (OL =AN amd 


Pe <= emNE Eelp &) -A ESP] 


Note thot fron problem T28 we hae E5lp) 70 
hile Es(p hd <0 cs well know. 

Them the expmenK = E(¢, 4) = En les) ~XEs(6) 
hao the property : 


f / | / 
Se = t —A Ss ) ion ( -) ( 
a Eol #4) ren Esle ie T¢g) -A HCW 


By choosing + thot yields Leg) =¢ we see thef 


P and % w (mg 40 RWG) <cc 


Home, ECR) a man mon } Gale 4)-A ESC? 70 
& o<eél 


amd 


fs- AH(U) << Finally ure hWaarg 
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é hapter ae 


8. | 
Por { +s sowrcd -USey pair - -~09 <UL 00, —20d VSO 


(2 —uz = (Q 
Q i) = e W/O : d tu WV) = We (u-vy> 


Jan o> a 
Choose Pm) , fr some Ay | PM iy) ae e ae 
Then (Pw) = =|_ py) es Cu) dy. 


whore &* = —\/aws . 
Chose Be + sakisty og +Py> =CQ . Then 


a 


ov (ae +e) 


i 


2 —\ 


4) : og Po” ols \* 2 
Ds! = we | Seeaae a (ara) o¢ | — we VQ = 6 
Rp?) = sd) + [ QMwLrMw du 


(i) Dae ut 
=-$-\ au) $n BI - soe du 


od) * 
=e as ; 0 < Od? < ue,> 
a 
ie wae 
S Ooo.) = s\n p® > O< Dee < wea Oy? 


: 1) ge 2 
ib) <i . (Q) = 
fe) . Ds 2 Wa Oy 
Hema Ds? = | ~Mas 5 eo <SE A Oy? 
eae Vaud? <5 £0 


Therefore fe sum distortion meacune » the vate destertiom 

Function ts Ytven tn terms ef paromder 620 a 

_ _ E- © 

De =p, min (8, we"), Rede) TE man (0,sbm ee) 
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ad 


@) D< mint Avy Aw jAr$ ‘ Then 
min (6, Ax) =6 , ae i128) Ayah Vi aja- bk. 
Thuwfere Do = & G-o . and 


he ERE 
2 > | 
tuted Lg tee LAF 
(b) 6= mon ¥ d\)r2~ AL 4 hom 
min (B.A) = wi mox (0, x dn AA) = 0 , eg ea Ver ce a 
Vhwo-fore De= Ls = a , and Ri(Dg)=0 


Md Ls 
= = ds iS the lenct elu tt Sor ohrch Ri(D)=0 


De =Dmay . 


ee Ae ae 
4B) = $+{ Qed) dea]B) aa = fs} QU) PLOY dcd]@) Aad 
Sinca ie P(ulwd¥ =|, Dclene (G,V; (B)= ye > AtGN)< dC a|B) 
* dL (Gd) > dle 
dB) = [mf Qua Pd QieyL |- Bas 1) I dadd se 
+ {. VQ Puta diai@) Sav; B) ddd. 
By defimtion = d QB) \- BWV Bd] =a , 
J} QuGPuchld) Bley. 1- SAB} badd 
= |) Qua PutdaacOd) dudu = fafa Pui | F deb] Ya dd 
_i< WA, a) nin Pea 
ge as oe & \| Q \% p (U\d) (V- — = ae p 
omd (.. Y aud Puan | dt dje\ dadd = (-( at 
= Naud) dolly = (| a. i 
< Sault Fiet}ae - 2 aL Jatin a] 
cee 2 30™ yr ~aLem™ rede 


le her 


239 


(s£cl 


6) a % DY + fl quincy) dB) B (G02) Ahad 
= zi 
js } Qaty Pu Pudi dierasad [ aud Pla) G1G.0 -Bydadd| 
pear A ae ae rae 
<2 EDP s (=I QudProid Buse daw | 


S; rN A\A rm : . ae 
NCS Jn{ Qed) POI) Seas Bydadv. ee ECR, p. Pe) 


ehee GU(Rip.Be) = -pR - Edm ({ | {pu aeatvy”™ ce 
Ya 


en 


= ie) 2] z= Dia dol Sf QUD PLOW S ch, TO)dudd | 


oo eo — 2: PES oy fe 
= TED + 3. (L/z) Ec OR, p, Be) 


3.4 
dy = Ef Uelarg} = lel o<f<i, fo, +1, £2, 
o ee 2 py She b sho © sh she 
P e° De = iy ee = 
} (a) ae. by res Ze as \ 
: | Fe \ oe ea ie 
Be eee eae a [eee randea 


Fur small 8<3 , ve. mnlO,. wl =6, vOel-27)] 
De = z\* mnl®, 6wNduo=8 
yp Ew Bol, Tapp 
R( Ds) = pune aiee Ba Ee a 
z ron ba) SS penta lcs 


\ _ ee 2 ae 
shige a [Latrareorais = Lbs 


x, Rip) = + Jn“ , ter De RP 


fae) 


we ee 


For awy discrete-time zero-mean stationary ergodic SOWELL 
arth spectral density Bw) and the arene distortim measure 


RCD) = sigs Rip), Rud) = Rild) 
where Rip) ts the rate distortim functtm obtamed ath ty 
tomsformatton ad =ul where coordivutes are Tndepemdynt 
From pase 490 we kmoo that ICPi)=TC( Pi), D(E)=D(PLD 


— nw 
and diCuv) a v) - 7 = C ug “Vay 


Yor Py U,c> (hi. D DCA) =D , we have Rio) TCR), 
A Ane ‘ie \ > a aoe eS 
Chsv se P_cv\a) = Ul taaap < (Ug @ Us) /2ap 


{=| 
2 > ee \ A 
AYA + ea LW Aa 
Them f | Gd Paya dv = pv + oo ra Ug 
A ; 


— feel. end Rid) < RC) < Auld) < FTCRD | 


Gut Lp) = 61 Gut Raya, Rapdady— 2 [| Bugle Pad 
ko) hi 
hone 5 
KD) & E(-1 Qicd) BLA) be Put) do Ao. 
afi. A slept _ (Ug- Bue) ae iy 
(-{ Gud Pcdldy LS [-Ldnaxay—- Ghee”) dag 
\ \ A = A 
= ~gda2apD — sap || qubs lh = av} ue 
an mga | In 2% BD —$ = -i dy ar.2aD 
4 0) a -Z ff Pub LPL ao 
Sma fo Dd =o, amd 


24! 


= |-[ Qua [ods £ Zor] du = adaptor cD 2a”, 
= \ 


Dp 2 Be eel 
Let PLY) UI Ga e i; - thom 


WD) < hOL) = bdnaxeo* = Ldaret ov) - 
~ Rep) <b LLP) = K(D) + h(DL) S KO) HHO YD 
= -1 da 2ead A Jraae (oD =-11,= 
Thenetore fer discrete time Stactomony SOUR , 
Ri) «ih = 


— 


4 
For = comtinuous- time stetimay SOWYCLS ohor2 
O(a) =0 for jwl > ao = H= 228 


We Knoo RODD < Re(D) where RalD) ic the rade 

distortion function for thi special caso of Gamssian 

source with same Dw) given parametrically - | 
De- + oe min 6, Pca dio == On minL & Ba] dio 


one R(Dg) = a \ mox 0, Im 222 dw = da \" mero, =I Jo 


And when Pia is Flak over lwo) < We oe gives moavkt mum 
rate distotim functim . (see Borger CIN) g- 134 D 
Thn @lwsk . Wwled 


Oo gt = 
T=) Belo = 7 ies - 2KB. 


o 


aA 


For SK. = Dg= adage te. S10, 


amd R(iDg)= ade bn ES = Biden 


AGL 


Therefore or band-limited continuous-time stationany 


sure R(D) = Re < BS 
Bb 

a a [sd+t f-fauwd ni du] @.3.10) 
Whon2 A 


o= ' Alu) : \-) Arty) Q Cu) ey <\.veY, } (8.3, W) 


Extend Lo +e NL whine N2|  mteyer. 


SS D) ae Amel, el sD + Tt | “f Quit) Qa Awe) du | 


Asn = Ay ly > Ssf Awe) Apluje see a) y rt 


N 
Mew chense u(y) = ae Tay TT a tun) QC) 


See a) eh Them 

Ru 6O) 2S een, 6 8D + ae rf QulW) {Le Qaty) 
_ She Nu (Un) + z WO un) du | 

a K{Uad — Ew heat) + Mf a LSD+ 


S40, Av éNgu 


= Sr h(n) - - h(Uc) + Rip) 


a 


ac l{ QW ba dtw dy | 


Rip) = ul Rin 6D) = ie a hiUwd} ae +Ri(D) 


a + hiUe) + ROD) 


“ 


3. 
1 From (7.1.44) amd @.3.\D 
RCDs) = h— WU) +RiCds) 
sd(2) 
= h-WUd) + WU) +805 — Wee da 
i sd 
= h+eDs -§" 2 Jo = Rip(D;) 4 
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cog er u) = Coa |B) exp [ -% uS. u) 
Lot [* ho +h, unitary metrix suok tut Br.=Ll Ac 
wath A eae Pe re Then 


Gay Ti Ey “exp (— e/a) = TT Quit) 


where Ag is the g xh ergomvalue Pare rae Homes 
TRU) = [hlte) = = 7 J-{ Qc) de Qu) Ad 
a = 2 [ Qe (Ka) bn Ba (ule) doe ae a Wie) 


oF 
= = = Im (ae rg) =thare rs Leh. 


By definvizoen ot hk, and applying Toepli 7 theorem , we hare 
hs Jinn, 4 thu) = hm [= Maze a 9 + re Ynds | 


L220 


= dhraxe +4. bin[ LE bade] = Share edd (EUG do 
\ 
= = Wome E | mhang Es opiag | dnt Be] da] yy 


BA ee 9 a Bw) 


oat wia-*{ a2 
A Pada 2a i+ (2y —_ : 
ae Loo SAC dd We A oe 
Xe 6 Seto ef > \ oO u* 
where = ton! 2 = — 
& a _& 
O(w* )= = l (sey > \+(= = & 8 a = I+ => p= Wy 
es a 
De= 2AJ.., mine, Sar} dw = HLowt*( 4 Peake] (oats trade E ») 


(= Duby Las (e sesetadh _ ao 6, a7] A tal 
A ar We ton! secZd \xp> AL al | 


L\* Gigs - tots) ] 
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\) 
rm) 
N 


ur \+ (Sy) de Bay ‘wl Mn L122] da 

= W C2u(4d -22 +2 tant 2 |" = wor} { plac i43) ap +2 tm 
(Dp) = 2 {mon 10, Jn BL da = oh (Lae Texle 
an] Walite?) a3 MC it(Z)*] dof 
==} Onl 4A?) — BoB Lal te*) + 2008 — ~2Wo tan’ a} 


— Ose fa 
\o 
wet ih Fev = yi.) 


i] oo 
4) (PJ) 
u(+) Se Wei aD a’ Pid 4 ost = 7 


ot 5 
S pid) FH = HS, peel , 
dit,s)= Efuuist , ost. set. i d(ds)frods =Afit), 
Gausston sourg with Sfruared err criterion: 
ik) Oe : — 
Se driuu)= 5 2 (uv) 
=| . 
—U7/ Je 


20 \ 
Qiw= Taw , Ww) = Tax & 


Q =f ua. Wao. Ved. © (a.ysB) =} Pt drujvy = dyCulB) 
Ot dy(uy) > &(ul@) 
dB)= (+f Qrwwdrc4lB) du = fof QW Privy) dr ¥1B) dudy 


= | Qi w Prcyl¥) Jcwi@[ 1- Gcu vB) + Bcwv:®)|dudy 
|-( iw vide) \- Faweridudy 
=f} Qr lu) Py (| 4) dr(u.v) dudy = \- | srl Tg 7 duty 


Ka we® (v]u) Cw-V) rdudv| 2 2 ak pv 
ee | oa | Mi ee -s 


J+) Quw Priwtufdyujpyl duu =} du [dujar) du 
= }-{ Quw de (wo)du = iF So (w= u®] du = owe g Z uJug 


Pa ean an +> 34 a es 


I 


lou W)Pr(v iu) Bu, v xB) dudy 
=}\[WRoeaveuy[f} ewer uw dy} og 
< pf \~| [ {-4 PrWQrulyyr*Fduy tia 


ort tT ert baf fT) Re Gruul *tantauh TT 
or ECR ¢ Ps) 


dy(B) < te wade [ \- {at Plulwehtu}Edodsl [Ja {OrWRulw Fea; paul 
Zee der L(G WPrwlu) Bruy-@ dud ‘le 


ee 


a 
1 
BD < +20"... [fawrawear Dah] 


| 20 
Ste D ni exp} -L-E, (Rp. P >| 
Using Samy arguement R617 2718) Ga) haure 
Av(p) < Se nmin (G8) + der expp-2 (-e-2= mae, Se 
Toe, amd using es ta theorem. 
Ce ri emis flo. 


Ge = fy Ol O (ire 
[306 ee Mend od Ete ree IR a | _ max max 0, rel sarees" ele 


E(R,Dg) 2 teen a PR+E,.(¢.8)], de> [3 (~ fey dol? 
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“We hawe that fur each €:>0, e>0, thy onits on mto 
No(@, R, €, €a) suck that tor ak 2 Ie thong enxrets a 
bock cade B sf ei R ond durattm T such toh 


d(B) = Dete,+ do erp }-E[ E(RDs)-e.]} 
wharw ER,Dp)>0 fr RD RUDg) 


AN 


eee 


ACB) = Z Quy dua) 
= : _s\ dle >Dd 
Refine indtcatar function #(43B)=}, dcujB)<D 


Using the facts: (\-B)+Fa|, dewBede, dua [l-d1 =p; 
d(Q) = = QniWdcyJB)CI-BowsBN + 2 Qui ¥) diulB)_P(u,B) 


S DAQuly) + do S Qylw SiuB) 


Averaging thts cver emsomble ef cacloc pleldc 


3B) J D+ db DA Qolw Fius@ < D+ de T Qulw> PLB) SP (usB) 
= B 


= Dido & Gulu) rf diwlB)>pluf < pa ge *PNER-R)+ ANI] 


A 


3.12 


eee 


Refine: Frey & Krep + é dn = -(1-Od0-8 -—€ bo 

F Ce) = + bCi-e) + — has = Ue) ti 

Fle) = bh co UG: ofec\” 

Sma Fl) <0, Ftd tw cemmver /. 

Fiey=0, Fo.3) = (C1) da6-7) — Gardyr = 0042 

¥. We knao fom umunen ity tht Fre) >o te scec.3 

€. Mie) >-eW fe o<ec .2 ; 
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2.\3. 


We knao on Pra (Q) grt -Ngl SNP E =| ter ang P20 
~ \y 
Bes uses ice (4) Jue =) 


amd lim o> NI | 
N2% [p-Ngl>Nns Cy ie so | 


Sine Dy < | et aoe 

im ree ae By Sn engrwr (4G eg =d 
“im Roar DEO 0 aaa wo tm men 
And since De < Ducgsry for oll 42 Nel eny 

wim ores ein Pe = Dcgay) vee tic! HO Dawg 


Buf we can csv se y>o as small oo we pleana oho N 
gees tity . Hence 


ly = Wolo e 2 
oe waigreny (0) 4 (-9) Dg = Ong =D. 
NY 
’ ~ 9 
7. pee 2 it cg" Dy =.) . 
B.A4. 


(4) EY =F oy he distinct CEMpOSTHTON of oudput SLE UOM CLS 
F layth NX -trom source olpbabst of sae A. 


A 
= Lyi + Tea V ALN =2,3,°--- 
\ - a 
ee ree ay ee Lesa ecunk 
2 2 \ 2 \ \ 
pe = Latkes = Lat L paring eas oe Pe Wen ge 


i 


=e (Ne) = NA oe CN 
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a 2 A) 
pM epetst ath LNT? ee a LL LG tle 
At (AA) +2424) = + A(A+\) < casi 


LA-= 


Suppose Li etna i oN ee 
ae ea STi aN 


Wad & shoo (nsi*teNA4 =< (nap! - 
(WD NS = ee 


SNe a 


ie anes 


b 
eee 
Bo={vi,va,-- ump of block luagth Nand re. R= la 
Chssen oitth probability 
N 
P(By) = Tf Py (Um) = Th TT PO Ua 
m=) 


m=\ n=l 
hans PM) = = Py Qu) 
ond PYcvlu) is the cumdrtoal probabr\sty Nreldiny te re 
Siste-tun uacdon ROD 0) eo 


Lemma 35,2’ 
Let am emsomble oS costlas hao selected according to pir Wy, 


Thom thang -exiéts am integar N* suck thd Ter omy N >* 
PJ dculBe) 7 Dy | we Gph)} = axp j-erpl N(S-ocny)] 
whing O(N) goes te Berm as fact ao INTN . 


Prost? Game ae (85,39) we hore 
Pr} dculBs) >Pe| wetaia)} = wep C-M Prfdini@yeds [Y etry} ] 


But PrfdulBy) <Pyl We Cytt)\ > exp L-N ER(Da:Q®) +0 6H) 
299 


C Sea Martin PAL TG, Appendix B) Fer seen N =N* > 
Pet d(ulBs) >Dg | 4 Cyl) 
<= op} —2Kp (NR) 2xp L-N} RCD: QY) + o(n)} 


2X p [- exp { NULR-R CDg: Qi) _ ocw)] 4 | 


= 


= 2KP ( —2xp nL § -<cn)]}) Q.E.D. 


Theorem 25.2 

There wots am mtagor N* sudh that tor hi N2N* 
ond amy composition Cult) thon orsts a ade By otth mR 
suck hat d(ulBo) = Dy Sor ofl We Cyl) 


whine De ws defmed to be R= RED: QW)t8 
proof: f 1 d&wMi@ 
oe us oi Pc u\Bs) = \ © dtu Bad SPe 


Thun 
= P(ulO) = D LZ PB) Fcy\B) 
¥eG)) B 


Wey (2) 
ws { diu}B)> Dy) we Zulth as | Cuts)\-oxp(—vrps NLS- otw]} 


os 
AEC 

The last mepalrty comes from absve lemma. Since (Culd| 

ts upperbsunded by AY, the # cof elements mm Un , 


utp} N [s-ocj}}] 


Z — By[Be) = AY evel 


YeGylt) 

= op ( NdnA —2xp§N [5-0cNI] ) 
Choose N* such the  expf N*CS- och IE ZN*IWA 
then S @culpr) <1 Foal NZEN*. 


Uu €Cy(Q) 
“There ts at least one code By for oth 2, Qiuldrd<I. 


And this tmplies thar BculBp)=o for all wet. & 


Sruivalently dcuj Be) = De Loy all WE Enld). Q.€.D0. 
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(C) For N2N%, a code satisfying dCu|Be) < Dp can 
he found fer each composi tim class Cyl). Using 
these Ly cedes, a Compestte code Pre Cm ke defined by 


me Ly 
(, = b By 
=| 
Tess Ws 4 Code ot = blotk length N arth bene Gade usords 
and rite Re=R ae dy La 


Frm) uly = CAD In (et) 
= “ Re = R +644) ORY 


Thus by choosing N large enough - the rads st Composite 
code Rc caw ko made arbitrary dose 4. Rk. 


a.19- 
PyCR,D) = min Pri d(ul@) >D1B}_ = Pr} dui) >D 


= & Ql) Pr{deujay>0] 4} 
From Lemma 2.5.1. we howe 
Pe¥ dcuj@y>p| 42 2 Pes N LR-RCD5Q’) + o(N)I 


Threfere 


y(R,D) = >a Qty) a 
— exp NCR — 1270Xx ROD >Q’) +o() | 
Qe’ 


—exp N TR—R(D>Q’) tot) J 


=e 
When R> Mx R(D>Q’) 
F(R.p) = —hm a dm Pal RD) 

<= |im wr MP NER mex R(D:G) + oN) a 


N00 


' FCRLD=Ha&. 


2s] 


(by) From Prob. 3.14 we know fer RCD) <R< MOx,, RID) 


EN) 
fucy 5 d(HiBy>D}< V {ue Ty? RD<RLD:Q")F 
Dy = Pref duj@>D|B> < Pref weko? diyjBy >0{ 


< Pr§ we Gui): RID) <RID:QM), 42h a-“Ly] 


< ea 
pes ees) Pf we CylVl 


< rl exp L-N 3(Q"), Q) tein) | 
— £2 Rip < RD: Qu) 


—v TQ", Q) + oN) 
Sal ees | 


me 
9: ROD) < ROD :Q) 
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